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PREFACE 


Тһе chief purpose of this volume is to present a general treatment of 
factor analysis including an objective basis for comparing various forms of 
solution. To accomplish this а thorough examination has been made of the 
logical and mathematical foundations underlying the methods of analysis 
employed. The treatment throughout has been made as rigorous as pos- 
sible with the hope that factor analysis would emerge as a sound branch of 
statistical theory rather than as a series of disconnected methods of appli- 
cation. А considerable amount of mathematies has been found necessary 
for such rigorous treatment. The text, however, has been made self-con- 
tained in large measure by the treatment of such subjects as higher dimen- 
Sional geometry and the fundamentals of matrix theory. 

Another aim of the text is the practical one of supplying the methods of 
analysis in a very simple, readable form for the worker who is not concerned 
with the rigorous foundation. Such an analyst would need only a general 
knowledge of the various forms of solution, as described in the text, and he 
could then follow the detailed outlines for computation, which are given in 
the appropriate appendixes. Complete checks on all the arithmetical proc- 
esses are also given throughout these appendixes. 

In Part I the analytic and geometrie bases for factor analysis are set 
forth. Certain fundamental statistics are introduced in Chapter II, the most 
important of which include the assumed factorial composition of variables 
апа the distinction between patterns and structures. Inasmuch as geometry 
is a convenient form not only for presenting the basic theorems of factor 
analysis but also for clarifying many ideas in the actual solutions, Chap- 
ter III is devoted entirely to that subject. The fundamental theorems basic 
to the various methods are given in Chapters III and IV. A new approach 
to the determination of the common-factor space is presented in Chapter IV. 

| In order to furnish an objective basis for selecting a given form of solu- 
tion, & set of statistical criteria are enumerated in Chapter V. When cer- 
tain of these standards are postulated, several “preferred” solutions are 
disclosed, These are presented formally in Chapter V, and the distinguish- 
Ing characteristics of each type are indicated in detail. The practical ana- 
lyst who does not desire to read the complete development of each method 
should become thoroughly acquainted with the properties of the types of 
Solution treated in this chapter (and also the oblique form of solution of 
Chap. XI) so that he can make an intelligent choice of а particular form. 
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Chapter Vis of funtlamental importance for a clear understanding of the 
problem of choice of a particular type of factor solution, and the synthetic 
treatment of this chapter should add unity to the subject of factor analysis. 

Тһе complete theoretical development of the preferred types of solution 
is given in Parts II and III. The theory underlying each form is presented, 
and this is followed by illustrative examples. Among the new methods in- 
troduced in the present text are the averoid modification of the centroid 
method in Chapter VIII and the objective procedure for oblique solutions 
of Chapter XI. Sampling formulas of factor coefficients and for residuals 
have also been developed in Chapter VI. Complete forms of analysis, in- 
cluding detailed steps of procedure for each method, have been given in 
Appendixes C, D, and E. These should be of chief value to the nonmathe- 
matical student who may find portions of the theoretical treatment some- 
what difficult. : 

The evaluation of factors and relationships among factors are dealt with 
in Part IV. A synthetic treatment of several methods for estimating factors 
from the observed variables is given in Chapter XII. The relative advan- 
tages and disadvantages of each method are indicated, so that the most de- 
Sirable procedure can be readily selected. A routine scheme for the estima- ' 
tion of factors is given in Appendix G. In the final chapter the relationships 
between two factor solutions obtained by different methods of analysis are 
discussed. Then, to assist the practical analyst in employing the various 
methods described throughout the book, a general outline of procedure is 
suggested in 13.4. 

The last part of the book consists of eight appendixes, the contents of 
some of which have already been indicated. In Appendix B a number of 
proofs of some of the fundamental theorems of the text are presented. 
These have been placed in the Appendix because of their mathematical 
complexity. These proofs are required for complete rigor, but the nonmathe- 
matical reader, who may accept the theorems without proof, can read the 
text without them. In Appendix H tables are presented for facilitating the 
caleulation of the standard errors derived in Chapter VI. 

The student whose background includes college mathematics through 
the calculus and a good course in statistics should be able to follow the en- 
tire treatment without supplementary reading. Since matrix theory may 
be unfamiliar, this topic has been treated in Appendix A. This contains the 
definitions, rules, and simple illustrations of the parts of this theory required 
for an understanding of the present text. The student who desires to learn 
only the forms of solution and the required routine analysis may omit much 
of the theoretical development and still have a good practical knowledge of 
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factor analysis. The instructor who uses this volume а а. text will probably 
want to exercise his judgment as to the portions to be omitted. It is sug- 
gested, however, that all basic theorems be considered and illustrated even 
though the proofs are omitted. 

No attempt has been made to give a historical account of the develop- 
ment of factor analysis. This has been omitted partly because of the greatly 
added bulk and partly because such an account would involve both statisti- 
cal method and psychological theories and thus would be outside the scope 
of the present general statistical treatment. For similar reasons it has also 
been found necessary to omit the treatment of a number of alternative ap- 
proaches to factor analysis such as the “Sampling Theory of Intelligence,” 
“Cluster Analysis,” and “Inverted Factor Analysis.” 

A number of practical illustrations of all the methods are presented 
throughout the text. Complete calculations are exhibited in order to clarify 
the various procedures. No additional exercises have been included, but it 
is planned to supplement the present volume with an exercise manual. This 
will include simple exercises, practical applications, and questions designed 
to increase the usefulness of the present volume as a textbook. For pur- 
Poses of reference a bibliography of contributions to the methods of factor 
analysis is also presented. This includes not only works on factorial meth- 
ods and some of their major applications but also related mathematical 
treatises, 

Each section in the text is distinguished by a number in bold-face type 
Consisting of the number of the chapter in which the section occurs, pre- 
fixed to the number of the section in that chapter and separated from it by 
à period; e.g., 2.10 indicates the tenth section of the second chapter. Simi- 
larly tables, figures, and equations are distinguished first with the number 
of the chapter in which they occur and then, separated by a period, with 
their serial number within the chapter; e.g., Table 7.3 refers to the third 
table of Chapter VII and “equation (0.2)" refers to the second equation of 
Chapter VI. А few page references are also included, but the above notation 
Will be found convenient for rapid cross-reference. " 
Тһе authors are greatly indebted to the Social Science Research Commit- 
e of the University of Chicago for grants which made possible the research 
required for the present volume. They are especially indebted to Professor 
Louis Wirth for his suggestion that the present synthetic treatment be un- 
dertaken and for his advice and encouragement during the lengthy prepara- 
tion of the manuscript. To the penetrating criticisms of Dr. А. L. O'Toole 
are due many improvements in the manuscript. The writers are also in- 
debted to Dr. L. В. Wilcox for criticism of Chapter III on geometry and for 
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his suggestion of the set-theory notations. To Dr. Lee J. Cronbach, who 
read several chapters of the original manuscript, they are grateful for helpful 
suggestions and criticisms. 

Karu J. HorziNGER 


Harry Н. HARMAN 
UNIVERSITY оғ CHICAGO 


May 1941 
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PART I 


ANALYTIC AND GEOMETRIC BASES 
FOR FACTOR ANALYSIS 


СНАРТЕВ 1 
INTRODUCTION 


1.1. Nature of Factor Analysis 


Factor analysis is a branch of statistical theory concerned with the reso- 
lution of a set of descriptive variables in terms of a small number of cate- 
gories or factors. This resolution is accomplished by the analysis of the 
intercorrelations of the variables. A satisfactory solution will yield factors 
Which convey all the essential information of the original set of variables. 
Тһе chief aim is thus to attain scientific parsimony or economy of de- 
scription. 

This aim should not be construed to mean that factor analysis necessarily 
attempts to discover the "fundamental" or “basic” categories in a given 
field of investigation such as psychology. It would be very desirable, of 
course, to base such an analysis upon a set of variables which measures all 
Possible mental aspects of a given population as completely and accurately 
as possible. Even in such a case, however, the factors would not be com- 
pletely fundamental because of the omission of important measures which 
were not yet devised. While the goal of complete description cannot be 
reached theoretically, it may be approached practically in a limited field of 
investigation where a relatively small number of variables is considered 
exhaustive. In all cases, however, factor analysis does give a simple inter- 
pretation of a given body of data and thus affords a fundamental descrip- 
tion of the particular set of variables analyzed. 

The essential purpose of factor analysis has been well expressed by Pro- 
fessor Truman L. Kelley: ‘‘There is no search for timeless, spaceless, popu- 
lationless truth in factor analysis; rather, it represents a simple, straight- 
forward problem of description in several dimensions of a definite group 
functioning in definite manners, and he who assumes to read more remote 
Verities into the factorial outcome is certainly doomed to disappointment." * 


1.2. Applications and Problems 


The applications of factor analysis up to the present time have been 
chiefly in the field of psychology, because the methods were invented by 
psychologists for dealing with certain of their problems. A factor analysis 

* “Comment on Wilson and Worcester's ‘Note on Factor Analysis, " Psychometrika, 
1940, p. 120. 
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may lead to some theory suggested by the form of the solution, and con- 
versely one may formulate a theory and verify it by an appropriate form 
of factorial solution. The latter approach is illustrated by Professor Charles 
Spearman's theory that “ай branches of intellectual activity have in common 
one fundamental function (or group of functions) whereas the remaining or 
specific elements of the activity seem in every case to be wholly different from 
that in all others."* He showed that if certain relationships (the tetrads 
defined in 4.3) exist among the correlations, all the variables can be re- 
solved into linear expressions involving only one general factor and an 
additional factor unique to each variable. These relationships furnish the 
statistical verification of the ““Two-Factor Theory." If a set of psychologi- 
cal variables yields correlation coefficients which do not satisfy the preced- 
ing relationships, then a more complex theory may be postulated. This may 
require several common factors in the statistical description of the vari- 
ables. 

Recently, Kelley has described a method involving factor analysis which 
may well be used in the selection and classification of army personnel.[ Of 
fundamental importance for this analysis is the selection of variables which 
have definite social significance. Two types of variables are involved: first, 
those which are measures of social utility and, second, those which are con- 
cerned with measuring the well-being of the individual. The former meas- 
ures are selected because of their bearing on vocational success, while the 
latter are chosen to reflect individual happiness. From such a set of vari- 
ables several factors are obtained. Then the profile of an individual can be 
compared with that of the average participant in the type of job being con- 
sidered. Kelley's primary purpose was not “‘to suggest the measures to be 
employed, nor even the detailed principles for their selection, but to give а 
method for utilizing them to their fullest . . . . in the service of a State dedi- 
cated to the preservation of human liberties and rights." 

Psychologists are planning experiments employing the techniques of fac- 
tor analysis to determine а small number of psychological tests to describe 
the human mind as completely as possible. The approach consists of a fac- 
torial analysis of a large battery of tests in order to identify a few common 
factors. Then the tests which best measure these factors, or, preferably, 
revised tests based upon these, may be selected as direct measures of the 
"factors of mind." However, only to the extent to which psychologists 


* "General Intelligence, Objectively Determined and Measured," Amer. J. Psychol., 
1904, p. 201. 


1 Talents and Tasks: Their Conjunction in a Democracy for Wholesome Living and Na- 
tional Defense, 1940. 


1 Ibid., p. 48. 
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agree that the tests selected are the "right tests" can they be said to be actu- 
al measures of the factors. Such "factor tests" should be of a “риге” nature, 
differing widely from one another so as to cover the entire range of mental 
activity. To the present time, several major studies* have been undertaken 
to identify factors from large sets of tests. The tests which best measure 
each of the factors, however, are still far from being “pure.” A great deal 
of research in test construction is still required before tests can be found 
which сап be regarded as even approximating the “factors of the mind.” 

The methods of factor analysis have been successfully applied, in recent 
years, not only in psychology but also in such varied fields as political sci- 
ence, business, and medicine. Sometimes, in the study of a particular prob- 
lem, variables from several different fields may be employed. Thus, in an 
attempt to discover the underlying causes for poor reading, psychological, 
physical, and medical evidence has been pooled for factorial analysis. In a 
widely different field a physicist has turned to factor analysis to discover 
some simple explanation of the intensity of cosmic rays under varying con- 
ditions of temperature and air pressure at different altitudes. A factor anal- 
ysis of stock prices of railroad equipment, automotive parts, and oils has 
been made to arrive at the fundamental indices of this type of business ac- 
tivity. It has also been proposed to employ factor analysis in medicine in 
order to obtain a simple functional classification of substances producing 
allergy. These are only a few of the types of practical applications that may 
be expected as the methods of factor analysis become more widely known. 

The foregoing applications of factor analysis are concerned mainly with 
testing hypotheses in scientific investigations. Another use, considered by 
some to be the chief value of factor analysis, is to supplement conventional 
Statistical techniques. By means of factor analysis the computation of mul- 
tiple correlation coefficients, partial correlation coefficients, and regression 
Coefficients can be greatly simplified when many variables are involved. In 
the problem of studying the relationship between two sets of variables factor 
analysis can again be employed. The best linear function of the variables in 
each set is obtained by factorial methods, and then the correlation between 
these composites gives what is known as the canonical correlation.t Such a 
Correlation is the maximum possible between the two sets of variables. This 
technique should be of great value in test construction, inasmuch as it fur- 
nishes the investigator with improved methods for obtaining more valid 
and reliable pools of subtests. 


* L. L. Thurstone, Primary Mental Abilities, 1938; K. J. Holzinger, Preliminary Re- 
Ports on Spearman-Holzinger Unitary Trait Study, Nos. 1-9, 1934-36. 


f Harold Hotelling, “Relations between Two Sets of Variates," Biometrika, 1936. 
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The present text is concerned primarily with the exposition of vàrious 
procedures in factor analysis. These methods are completely illustrated by 
numerical examples taken from psychology, biometry, and political science. 
These examples are used chiefly to clarify the theoretical treatment rather 
than to exhibit the practical usefulness of factor analysis in the broader type 
of application mentioned above. 


1.3. Scientific Theories 
Factor analysis, like all statistics, is а branch of applied mathematics. 
It employs observed data, which necessarily include discrepancies, and 
hence may lead to several different mathematical formulations of a given 
problem. In general, an applied mathematical theory consists of the follow- 
ing parts: 
SCIENTIFIC THEORY IN А GIVEN FIELD 
1. Observed data (with discrepancies) 
2. Mathematical theory 
Postulates 
Definitions 
Theorems 
3. Verification with data 


Any such applied science involves some mathematical theory as its basis, 
plus a set of observations, and, finally, the check of the particular mathe- 
matical theory with the original data. In all problems of applied mathemat- 
ics there may be a number of mathematical theories which explain the phe- 
nomena in a satisfactory manner. A misunderstanding of the relationship 
between a mathematical theory and observed data is frequently encoun- 
tered. When a theory has been successfully employed in describing a set of 
data, there is a tendency to accept this law as the only correct one for de- 
scribing the observations. 

Furthermore, it is sometimes inferred that nature behaves in precisely the way 
which the mathematics indicates. As a matter of fact, nature never does behave in 
this way, and there are always more mathematical theories than one whose results 
depart from a given set of data by less than the errors of observation. 

The danger is always when a theory has been found to be convenient and effective 
over a long period of time, that people begin to think that nature herself behaves 
precisely in the way which is indicated by the theory. This is never the case, and 
the belief that it is so may close our minds to other possible theories and be a serious 


impedence to progress in the development of our interpretations of the world 
around us.* 


*G. A. Bliss, “Mathematical Interpretations of Geometrical and Physical Phenome- 
па,” Amer. Math. Monthly, 1933, pp. 472, 477. (Italics ours.) 
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The above observations may be illustrated in several different fields. One 
of the simplest cases arises in the problem of surveying a small tract of land. 
For this purpose either of two mathematical theories, plane or spherical 
trigonometry, may be applied. Thus, in surveying а city lot the results by 
either theory would be equally satisfactory, and the engineer would prefer 
the plane theory because of its greater simplicity. In this instance, how- 
ever, there is no doubt as to the greater accuracy of the spherical theory 
Since the earth is essentially spherical. 

In the field of astronomy there are two theories describing the solar sys- 
tem. The Ptolemaic and Copernican theories, with suitable modification of 
the former, describe the motions of the planets with equal accuracy. “There 
is really no advantage for either of these theories as compared with the 
Other, as far as their adaptability to explain numerically the facts of the 
solar system is concerned. The Copernican theory is, however, much the 
simpler geometrically and mathematically. For this reason it has been 
adapted and developed until astronomers can predict coming celestial 
events with most surprising accuracy.’’* 

Even the subject of geometry, which might seem to depend on a unique 
mathematical theory, can be described by means of many different theories. 
Thus, the physical configurations in a plane can be interpreted in the light 
of Euclidean geometry, Riemannian geometry, or various other types of 
non-Euclidean geometry. Therefore, the applied science of geometry can 
have several alternative theories as its basis. 

As in the foregoing illustrations there are different theories, or forms of 
solution, which may arise in the factorial analysis of a particular set of 
data, The usefulness of factor analysis as a scientific tool has been ques- 
tioned by some workers because of this indeterminacy. It should be evi- 
dent, however, that this is tantamount to indicting all applied sciences 
because they do not depend upon unique theories. 

Since the beginning of this century, psychologists and statisticians have 
developed several types of factorial solutions. The proponent of each sys- 
iem of analysis has urged its suitability for the interpretation of psycho- 
logical data. These varying points of view have been wittily described by 
Cureton as follows: 

Factor theory may be defined as a mathematical rationalization. A factor-ana- 
lyst is an individual with a peculiar obsession regarding the nature of mental ability 
Ог personality. By the application of higher mathematics to wishful thinking, he 
always proves that his original fixed idea or compulsion was right or necessary. In 
the process he usually proves that all other factor-analysts are dangerously insane, 
and that the only salvation for them is to undergo his own brand of analysis in 


* Ibid., pp. 477-78, 
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order that the true essence of their several maladies may be discovered. Since they 
never submit to this indignity, he classes them all as hopeless cases, and searches 
about for some branch of mathematics which none of them is likely to have studied 
in order to prove that their incurability is not only necessary but also sufficient.* 


Тһе factor analyst must recognize the fact that methods other than his 
own must have some merit. This attitude can be expressed in a manner 
analogous to that employed by Barclay in his discussion of the various bid- 
ding systems in contract bridge.f The argument in the following two para- 
graphs is essentially that of Barclay with the phrases “bidding systems" re- 
placed by “types of factorial solutions" and "players" by "analysts." 

It is a positive fact that every one of the types of factorial solutions in 
extensive use today is an efficient one. When known thoroughly, a skilled 
analyst can apply any one of them to produce fine results in a desired situa- 
tion. The enthusiastie partisan who depreciates the efficacy of some other 
method and calls it à poor one thereby furnishes indisputable proof of one 
of two things—either his own sincerity or his own ignorance. Not just ig- 
norance of the details of the other method, but general ignorance. Any per- 
son of intelligence should know that it is highly improbable that any of the 
brilliant psychologists and statisticians who have worked on these problems 
for the last thirty years would produce and advocate a “rotten” system. 

The analyst who learns all the leading factorial methods and becomes 
familiar with them in actual use will be able to defend any method that 
someone may endeavor to ridicule. When some authority makes light of 
another method of analysis, it is quite likely that the man advocating a par- 
ticular system is extremely earnest in believing that he is offering his fol- 
lowers the very best procedures, but it may also be that the man has not 
given enough study to the other's method to realize its fine points as well 
as its deficiencies. Since no one factor method can cover all types of data 
which the analyst might like to describe, he should be familiar with all the 
methods so that he can select that one which is most suitable in а, particular 
problem. 

The various types of factorial solutions correspond to the different mathe- 
matical theories in the description of a particular scientific problem. Five 
preferred forms of solution are presented in the text, any one of which may 
be selected by the analyst for dealing with a given body of data. 'The exposi- 


* Edward E. Cureton, “Тһе Principal Compulsions of Factor-Analysts," Harvard 
Educational Review, 1939, p. 287. 


f Shepard Barclay, Win at Contract with Any Partner, 1933, pp. viii-ix. 
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tion of these methods and their comparison are the primary aims of the 
present text. 

The properties and statistical criteria underlying each of the preferred 
types are presented in Chapter V. These standards reveal the salient fea- 
tures of the preferred solutions, indicating their advantages and limitations. 
When an analyst has selected a particular form of factor solution for the 
description of a given problem, he should realize that there are other meth- 
ods which might afford an equally satisfactory interpretation of the data. 


CHAPTER II 


BASIC STATISTICS 
2.1. Introduction 


In this chapter certain elementary statistical notions are introduced. 
These ideas are essential for much of the subsequent analysis and are there- 
fore presented in detail for the sake of rigor and completeness. The defini- 
tions are so formulated that they are applicable in any field, such as educa- 
tion, psychology, or commerce. Consequently, the formulas based on these 
definitions are general in their application. 

After the preliminary definitions of some elementary statistical concepts 
have been presented, the composition of a statistical variable is treated. 
Then a fundamental distinction between two necessary parts of a factor 
solution, pattern and structure, is developed. A brief discussion of the statis- 
tical adequacy of a factor solution follows. The general arbitrariness of any 
scientific theory is exhibited here by the indeterminacy of factor solutions. 
Finally, methods of organizing data for subsequent analysis are presented. 
In illustrating these methods a set of psychological variables is employed. 


2.2. Statistical Variables 


Any statistical study involves a selection of a group of N individuals 
with a number of common attributes. In a particular investigation the 
group of individuals might consist of such objects as persons, schools, cities, 
or commercialenterprises. Various aspects of such individuals may be meas- 
ured and referred to as statistical variables. These will be designated by 
X; (j —1,2,...,n), or more briefly 7, where n is the number of variables 
in aset which is employed in describing the N individuals. Particular values 
of a variable X; for individual 4 will be represented by Ху. These will be 
called observed values inasmuch as they are measured from an arbitrary 
origin and by an arbitrary unit. 


The sum of all N values for а variable X; is written in the form 


N 


Уху, 


ісі 
but this may be put more simply in the form 
ZXj, 
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Where it is understood that the summation extends over all values of the 
variable. This convention for the summation with respect to the number of 
observations of a particular variable will be observed throughout the text. 
Furthermore, the index i will be reserved explicitly to refer to the individu- 
als, that is, for the range 1 to N. The mean of the N values of X; is then 
given by 


Derived values of each variable can be obtained by fixing the origin and the 
unit. When the origin is taken at the mean, a particular value 


Tji = Ха- М; 


will be called a deviate. The standard deviation of a variable X;jis defined by 


(2.1) oj = МЕЗ 


where, of course, the summation is over the N values of i. By taking the 
Standard deviation as the unit of measurement, standardized values may be 
defined as follows: 


(2.2) Bj 2%, 
cj 
The set of all values z; (i = 1, 2,..., N) will be designated as a statistical 


variable z; in standard form. Obviously the variance (standard deviation 
Squared) of z; is equal to unity. 

The product-moment coefficient of correlation for any two variables j and 
k is defined by 


(2.3) op O 


(2.37) e - Хтнть _ Zeus : 


The intercorrelations between all possible pairs of variables in a set con- 
stitute the basie data for factor analysis. It will be assumed here, as in all 
other cases where the product-moment correlation coefficient is employed, 
that it is most useful in case all regression lines are sensibly linear and all 
variables are normally distributed in the population and approximately so 
In the sample. 
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2.3. Composition of Variables 


The simplest mathematical expression for describing a variable in terms 
of several others is a linear one. This form is the one frequently employed 
in ordinary regression methods, although more elaborate expressions are also 
used for а small number of variables. For a large set of variables, however, 
а simple approximation becomes almost imperative, and hence linear ex- 
pressions are used. Similarly, in factor analysis the simple linear form is 
taken to represent a variable z; in terms of a number of underlying factors, 
or hypothetical variables, which are taken to be in standard form for con- 
venience. А factor may be said to be in standard form if in the population 
its mean is zero and its standard deviation is unity. Furthermore, it is as- 
sumed that these properties also hold for a given sample. The factors are 
also assumed to be normally distributed. 

Several types of factors may be employed. Common factors are those 
which are involved in more than one variable of a set. Two instances of 
common factors arise: a general factor, present in all variables, and a group 
factor, present in more than one but not in all variables. A factor which 
appears exclusively in the description of a particular variable is said to be 
unique. Common factors are necessary in order to &ccount for the inter- 
correlations of the » variables. Each unique factor represents that portion 
of a variable not ascribable to its correlations with other variables in the set. 

Employing the notation Fy, Fo, . . . , Fm for m common factors, and Ui, 
Us, ... , Un for the unique factors, the complete linear expression for any 
variable z; (j = 1, 2, 3,..., n) may then be written in the form 
(2.4) 2} = ал + agFa + азЕз +... + аа”, + ajU; , 
where the prime is used to indicate the theoretical form of the observed 
variable. For simplicity the prime will usually be dropped hereafter in the 
linear expression for the variable. There are, of course, n equations of this 
form—one for each of the n variables. Equation (2.4) might be written 
explicitly for the value of variable 2) for a particular individual 7 (2 -- 1,2, 


135 pd IOs 
(2.5) Zii = ааР + agFa + аза +... 4 аР: + а; ы. 


In this expression it is assumed that there аге N values of each of the fac- 
tors, corresponding to the N individuals of the sample. The basic problem 
of factor analysis is to determine the coefficients, aj, . . . » Aim, ОҒ the com- 
mon factors. This determination is made by the various methods described 
in Parts II and III, employing the observed intercorrelations of the n vari- 
ables. After these coefficients have been determined, the factors for the 
individuals can be evaluated by the methods of Chapter XII. 
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The contributions of the factors to the total variance of any variable 2; 
will next be shown. To express the variance of z; in terms of the coefficients 
of the factors, square both sides of (2.5), sum over the N values of the re- 
Spective variables, and divide by N, and obtain, upon expanding, 


222; ZF? ZFi ZFii ZU: 
у “Фу they уу 
УЕ Fo: EP il mi 
+2 (anan N eee FO mam NS ae 
ZFuU; >Ғ, ОУ; 
+ ала; N : Toc ата; MÀ 8. 


Now, since the variance of a variable in standard form is equal to unity, 
апд all variables (including the factors) are assumed to be in standard form 
for any sample, the last equation may be written 


же = о =a}, +a +... + а, +a} + (алалы, 


+... + Ginüirg,v;) - 
If the factors are uncorrelated,* this expression simplifies to 
(2.7) = 63, = a}, tai, +... + ada. 


The terms on the right represent the portions of the unit variance of z; 
ascribable to the respective factors. For example, a?, is the contribution of 
the factor F, to the variance of z;. The total contribution of F to the vari- 
ances of all the variables is given by 


а + а + а +... + ah, 


n 
and, in general, the total contribution of a factor Р, is defined to be S аў}, : 
7=1 
For any variable 2; the sum of the squares of the common-factor coeffi- 
cients is known as its communality h?. The last term in (2.7), which is the 
Contribution of the unique factor, is called the uniqueness of z; and indicates 
the extent to which the common factors fail to account for the total variance 
9f the variable. The uniqueness of a variable may be separated into two 
Portions due to the selection of the variables and their measurement. If 
additional variables are added to a given set, their correlations with the 
latter may make it necessary to postulate further common factors. For & 


* The case of correlated factors is treated in Chap. XI. 
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fixed set of variables such potential linkages of any one of them with others 
can be expressed only as a portion of its uniqueness. This portion is termed 
the specificity of the variable. The remaining portion of the uniqueness may 
be ascribed to imperfections of measurement. Тһе consisteney with which 
а variable measures whatever it measures is called its reliability. Тһе re- 
liability of а variable may be expressed by the correlation of repeated meas- 
urements or of parallel forms of the variable. In either case, the two evalu- 
ations will be represented by z; and 2, and their correlation rj; will be 
called the reliability of the variable. 

When the unique factors are conceived as decomposed into the two types, 
described above, the expression (2.4) for any variable may be written in 
the form 


(2.8) Zi = aii + apat... 4 аһ Ға + biS; + cjT'; 
@=1,2,...,%). 


Неге 8; and T; are the Specific and unreliable factors, respectively, and b; 
and c; their coefficients. Now, if the variance of 2; is obtained in terms of 


the coefficients of (2.8) and compared with the expression (2.7), it is evi- 
dent that 


(2.9) a} = bF + о. 


From the preceding analysis it is apparent that the total unit variance 
of a variable may be resolved into three portions, attributable to common, 
specific, and unreliable factors, The first portion has been called the com- 
munality, while the last two portions of the variance of any variable have 
been named specificity and unreliability. On combinin 
unreliability, the uniqueness is obtained; or, again, the sum of the commu- 
nality and specificity gives the reliability of a variable. A recapitulation of 
these ideas may be expressed in the following form: 


bj = specificity қ S; — specific factor ; 


d= unreliability , T; = unreliable factor , 
(2.10) {№ = 229, = communality , F, — common factor у 
8-і 


аў = B+ = uniqueness = 1 — М, 


hy + b} = reliability = 1 — G = ту, 

By factorial methods the communality h? and the uniqueness a? of a set 
of variables are obtained. The uniqueness of each variable may be split 
into the specificity and unreliability, but this is independent of the factorial 
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solution and follows such a solution. If the reliability туу of a variable z; is 
known (it may be obtained by experimental methods), then the unreli- 
ability may be obtained by means of the equation 


(2.11) б-1-ту. 

"Then, of course, knowing the unreliability, the specificity is given by the 
“Pythagorean”’ relation (2.9), or 

(2.12) b =а}—с], 

Where the uniqueness a is known from the factorial solution. From the 
last expression in (2.10) it follows that 

(2.13) М-ти- 0 Sry. 

In other words, the communality of any variable is less than ог equal to 
the reliability of the variable, and equals the reliability only when the 
Specificity vanishes. 


An index of completeness of factorization may then be expressed in the 
following form: 


Бк ‚ = 100K} _ 199 сошшшзашу 
) H; B 100 ^ "liability 


This index shows the percentage of the reliability variance accounted for 
by the common factors. If it is desirable to find the total communality, 


n 
a М, instead of the separate communalities for each variable, and also the 
7=1 


total specificity, then the complete indez of factorization, which is defined by 


(2.15) re px 


7=1 


тау be obtained. The index Н plays the same role for the total set as H; 
does for the individual variable gj. 

The index H j is always less than 100 and approaches 100 only when the 
Specificity b? vanishes. Any method of analysis for determining the coeffi- 
Cents a;, should not be carried to the point where no specificity is present 
When it is known that some specificity exists. 

Some workers may not care to assume specific or even unreliable factors 
as Indicated by (2.8). In this case the factors S; and Т, are not postulated, 
апа the number m of statistically significant common factors may be less 
than, or equal to, the number of variables n. In the opinion of the authors, 
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however, the hypotheses of factors indicated by (2.8) appears most tenable 
even for variables which appear to describe a set of objects very completely 
and with great precision. 


2.4. Factor Palterns and Structures 


The factor problem will now be dealt with more generally, and for this 
purpose correlated factors will be postulated. The uncorrelated factors may 
then be considered as a special case of these. A set of equations of the form 


24 = ayF, d аР, +... + а,„Р, + aU, 
(2.16) 2; = aj F, + аР, +... + doml'm T a,U, 


2, = An, + as FP +... + CN m +a,U,, 


expressing т variables in terms of m common factors and n unique factors, 
will be defined as a factor pattern, or, more briefly, pattern. As indicated 
above, the common factors Р, (s = 1, 2,..., m) may be correlated or un- 
correlated, but the unique factors U; G = 1, 2,..., п) are always as- 
sumed to be uncorrelated among themselves and with all common factors. 
In the linear description of a given variable any number of common factors 
(и € т) may be involved. This number is called the complexity of the 
variable. 

Factor analysis yields not only patterns but also correlations between 
the variables and the factors. A set of such correlations will be defined as 
a factor structure, or merely a structure. Both a structure and a pattern are 
necessary in order to furnish a complete solution. The functional relation- 
ships between the elements of a structure and the coefficients of a pattern 
will now be shown. 

Multiplying any one of equations (2.16) by the respective factors, sum- 
ming over the number of observations N, and dividing by М, produces 


Тағ, = ад + аж кк, +... 4 Geter, +... + Amr A у 
Tae, = ФР, + ap +... agree, +... + аттыға» 
(2.17) 
Тағ, = Qarr, + айтғ,р, +... + а, Ф... Qim? РР, y 
Тағ, = АЛЕ ғ, + ағы, +... + GuTE.E, +... + dim, 
and 


(2.18) Tzu; = Qj. 
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Equation (2.18) shows that the elements т. v; of a factor structure are 
the coefficients of the unique factors in the pattern. When no confusion 
can arise, the table of correlations of variables with common factors only, 
i.e., the table of 7;r,, Will be referred to as the factor structure. The ma- 
trix formulation of the relationship between a fector pattern and structure 
is given in Appendix B.11. 

In а numerical problem equations (2.17) may be used to evaluate the 
correlations of the variables with the factors when the coefficients of the 
pattern and the intercorrelations of the factors are known. More frequent- 
ly, however, these equations are used to obtain the values of the pattern 
coefficients when the correlations between variables and factors and the 
correlations between the factors themselves are known. 

Formally the equations (2.17) may be considered as constituting n sets 
of m linear equations іп the unknowns а, (j = 1, 2,...,; s= 1, 2, 

. , m), with the left-hand members as known quantities. It is then pos- 
sible to solve* for the m coefficients ау in each of the n sets, that is, for each 
value of j. Here it has been assumed that the elements of the factor struc- 
ture and the correlations between factors are known, and so the coefficients 
of the factor pattern may be obtained. The result is 


(2.19) "T 1 


Where A is the determinant of coefficients of the а» and Aj, is the determi- 
nant obtained from A by replacing the elements of the sth column by the 
elements Тек) Тауруз + у Te It has thus been shown that a pattern can 
be constructed from a known structure. 

It is interesting to notet that the values aj, in the above solution turn 
Out to be regression coefficients, so that equations (2.16) are in the form of 
Tégression equations with the errors of estimate included. These errors are 
Tepresented by the unique factors. A plus-or-minus sign may be placed in 
front of the à; because the unique-factor coefficients in a pattern are usually 
obtained, after all the other coefficients are known, in such a manner as to 
bring the variance of each variable up to unity. This procedure involves а 
Square root and thus leads to the indeterminacy of sign. 

Since the unique factors are of minor interest in factor analysis, it is con- 
Venient to write that portion of a factor pattern which involves only the 
Common factors. For similar reasons, a table of coefficients of the common 
factors may be referred to as a factor pattern, when there is no ambiguity. 


n * Maxime Bocher, Introduction to Higher Algebra, 1935, p. 43. See also Appen. б.1 
9r à simplified method of. solving a set of simultaneous equations. 


1 See Appen. B.2. 
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From equations (2.17) it is apparent that the elements r.,r, of a struc- 
ture are generally different from the coefficients a;, of a pattern. In case 
the common factors F, are uncorrelated, that is, re, z, = 0(5 > 4), then equa- 
tions (2.17) reduce to 
(2.20) Tar, = Ais (1-<1,%,.2, Me oH 1, 2>,, 70% 
Thus, only in the case of uncorrelated factors are the elements of a structure 
identical with the corresponding coefficients of a pattern. In an analysis in- 
volving uncorrelated factors, a complete solution is furnished merely by а 
factor pattern inasmuch as the correlations of the variables with the factors 
are given by the respective coefficients. 

As already indicated, both structure and pattern should be produced in 
making a complete factor analysis. The structure reveals the correlations 
of variables and factors, which are useful for the identification of factors 
and for subsequent estimates of the latter. The pattern shows the linear 
composition of variables in terms of factors in the form of regression equa- 
tions. It may also be used for reproducing the correlations between vari- 
ables to determine the adequacy of the solution. In comparing different 
systems of factors for a given set of variables, patterns are again useful. 


2.5. Statistical Basis for Adequacy of Solution 


The manner in which a set of factors accounts for the intercorrelations in 
a set of variables will be considered next. The variables are assumed to be 
linearly composed of the factors as given by the pattern (2.16). Since the 
factors are assumed to be in standard form, the correlations between vari- 
ables may then be reproduced from the factor pattern by the following pro- 
cedure: multiply any equation (2.16) of a variable by the equation of an- 
other, sum over the number of individuals N , and divide by N. For any 
two variables z;, z; (j, k = 1, 2,... , ^) their correlation may be written 


in the form* 
Tik = алап + азаа + аза 4... + QjmOkm 
+ (ала + ana;j)rr, r, + (алаа + QmQja)Tryr, +... 
(2.21) “(ала + ама js)T rr, + (аракз + аманутғғ, К... 
+ (aito + aram) TE ea Б... H anar u, + Quarry; t+: 
F аак, F атаву; + айти шк, 


* Holzinger and Harman, “Relationships between Factors Obtained from Certain 
Analyses," Journal of Educational Psychology, 1937, p. 324. 
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where the reproduced correlation (computed from the pattern) is written 
тү, to distinguish it from the observed correlation r;,. This distinction will 
be made throughout the text. 

The unique factors have been assumed to be uncorrelated with the com- 
mon factors and among themselves, hence Tru; = Tru, = Tuy, = 0. If 
the common factors are uncorrelated, equation (2.21) simplifies still further. 
Тһе correlations Try, (S = t; s, t = 1, 2, . . . , m) are then zero, and every- 
thing below the first line of the equation vanishes. For the case of uncor- 
related common factors, any correlation is thus reproduced from the factor 
pattern by an equation of the following form: 


(2.22) THe = апаа + js +... + оњак 
G = k; j,k =1,2,...,n). 


This expression is merely the sum of the products of corresponding pattern 
coefficients of the two variables correlated. 

Equation (2.22) may be represented, perhaps more simply, by use of 
matrix notation.* Let the matrix of coefficients of the common factors of 
the pattern be represented by A, that is, 


ад Gu @3 ... Aim 
ву аљ og ... Com 
(2.23) А- Р 
ад ав аӊ... а, 
аз An Ang +++ Anm 


which will be referred to as the pattern matrix. Then the product of A by 
its transpose A’, i.e., the matrix obtained from A by interchanging its rows 
and columns, is the matrix of reproduced correlations Rt. This result may 
be put in the formt 


(2,24) R! = АА”, 


* For an elementary account of the theory of matrices which is essential for the pres- 
ent text see Appen. А. 

Т Equation (2.24) has been called “Һе fundamental factor theorem” by L. L. Thur- 
stone (Vectors of Mind, 1935, Chap. 11). In the case of correlated factors, which will be 
treated in Chap. XI, the expression corresponding to (2.24) is 


“Rt = Add’, 
where ф is the matrix of the intercorrelations of the common factors. This relation was 
rst presented, in expanded form, by Holzinger and Harman, op. cit. More recently, it 


was formulated in matrix notation by Tucker, “Тһе Role of Correlated Factors in Factor 
Analysis,” Psychometrika, 1940. 
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or, writing the matrices in detail, as follows: 
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2 , , 
| hi т fag Tin 
| ra М т; Tan 
| Y T ex 
| P | 
| , 2 , 
|Та Тіз № Tin 
| : 
| , 
Ті) Tre TL h | 
Rt 
| an ав аљ | 
| а 4% dom | ай аш ад Ani 
= | M ja «| ај ат а» ал? 
| | 
|an ie аж | [ERE TE 
| i | 
| | 
B | Ham Am Aim бз 
|| аа аш Gam 
= A A 


The correlations (2.22) are merely elements in the matrix Rt. The diagonal 
elements А? (j = 1, 2, . . . , n) are the communalities of the respective vari- 
ables and may also be obtained by setting j = k in equation (2.22). It may 
be noted from (2.21) that the algebraic signs of all the entries in any column 
of a pattern matrix A may be reversed without affecting the reproduced 
correlations. 

Evidently the factor problem is concerned with the correlations of a set 
of variables. These correlations are reproduced, when a pattern of the form 
(2.16) is assumed, by means of the coefficients of the common factors. The 
diagonal elements of Rt should also be reproduced from this portion of the 
factor pattern. Thus if, in a matrix of observed correlations, numbers ар- 
proximating the communalities are put in the diagonal, the factor solution 
will be of the form (2.16) and the observed data (including the diagonal 
entries) will be closely reproduced by equation (2.24). On the other hand, 
if unity is in each diagonal cell of the observed correlation matrix, the factor 
solution necessarily would involve only common factors in order that equa- 
tion (2.24) may reproduce ones in the diagonal. In this case there is obvi- 
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ously no allowance made for unique factors. If other values between the 
communalities and unity (such as reliabilities) are employed, then the form 
of solution would involve common and unreliabilit y factors, but the specific- 
ity would be included in the common-factor variance. From these consider- 
ations it is clear that the values put in the diagonal of the observed correla- 
tion matrix determine what portions of the unit variances are being fac- 
tored. 

The general problem of relating the reproduced to the observed correla- 
tions, regardless of the diagonal entries, will be considered next. The correla- 
tions reproduced by the factor pattern, as given most generally in (2.21) or 
for the case of uncorrelated factors in matrix Rt, should not agree exactly 
with the observed correlations because allowance must be made for sampling 
and experimental errors. It is a commonly accepted scientific principle that 
& theoretical law should be simpler than the observed data upon which it is 
based, and hence discrepancies between the law and the data are to be ex- 
pected. In the case of factor analysis, functions (the correlations rj,) of the 
assumed linear composition of variables should be expected to vary some- 
what from the observed values. 

After a factor pattern has been obtained, its adequacy as a description 
of the intercorrelations of the variables is determined by "removing the 
factors." This is done by forming the correlations from the pattern as given 
by equation (2.21). These values are then subtracted from the correspond- 
ing observed correlations, and the resulting differences are known as residual 
correlations. Such residuals are defined by the following formula: 


(2.25) Fie = ik — Tie, 


where rj, is the observed correlation and rj, is the correlation reproduced 
from the pattern. 

In case the common factors are uncorrelated, it has been shown that the 
lengthy expression (2.21) for reproducing the correlations reduces to the 
simple form (2.22). The residuals then reduce to the form 


(2.26) # = Тук — (адаа + аваз +... 4 ажа»). 


The question then arises as to how nearly the correlations reproduced 
from a factor pattern should fit the observed ones.* The agreement may 


* Up to the present time a good standard for “when to stop factoring” has not been 
developed. This problem is discussed in Chaps. V1, VII, and VIII. 
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be judged by the size and distribution of the residuals 7;,. The magnitude 
‘of the residuals should, of course, be approximately zero. When all com- 
mon factors have been removed in forming the residuals, then no further 
linkages between variables exist. It might, therefore, be expected that the 
distribution of residuals would be similar to that of a zero correlation in a 
sample of equal size. The standard error of such a zero correlation is given 
by the formula 


di mrt 
or, since N is usually large, 
(2.27) драм 
5 т=0 VN * 


А standard for judging adequacy of fit then may be taken to be 


(2.28) 


oF 


ША 


= 
VN’ 


where е; is the standard deviation of the series of residuals.* This standard 
is a tentative one since it depends only on the size of the sample, whereas 
experiments have shown that the size of residuals depends also on other 
characteristics, especially the number of variables. More exact sampling 
error formulas will be developed in Chapter VI. 

On the basis of the size of the sample alone, the following conclusions from 
the criterion (2.28) may be drawn. If оғ is appreciably greater than 1/ VN, 
it may be concluded that there are further significant linkages between 
variables, and a modification of the form of solution is required. In case оў 
is considerably less than 1/ VN, it would appear that unjustified linkages 
between variables have been included in the solution. When the standard 
deviation of the residuals is just below that of a zero correlation, the solu- 
tion may be regarded as acceptable in the light of the above standard. A 
standard equivalent to (2.28) has been given by Kelley, and substantially 


the same basis for judging the adequacy of fit of a factor solution has also 
been given by Thurstone. ї 


* An alternative value for о?, 
227? 
тп — т+1)` 
j Truman L. Kelley, Essential Traits of Mental Life, 1935, 

Ор. cil., p. 147. 


taking degrees of freedom into account, is given by the 
formula «о; = 


p. 12, formula (7). 
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2.6. Indelerminacy of Solution 


In any scientific field the observed phenomena ean be described in a great 
variety of ways which are mutually consistent. The choice of a particular 
interpretation must then depend upon its utility. This arbitrariness or in- 
determinacy of description has been recently emphasized by F. R. Moulton 
in comparing various approaches to the measurement of the velocity of 
light. In conclusion he says: 

All this illustrates Poincaré’s thesis that every set of phenomena can be interpret- 
ed consistently in various ways, in fact, in infinitely many ways. It is our privilege 
to choose among the possible interpretations the ones that appear to us most satis- 
factory, whatever may be the reasons for our choice. If scientists would remember 
that various equally consistent interpretations of every set of observational data 
can be made, they would be much less dogmatic than they often are, and their 
beliefs in a possible ultimate finality of scientific theories would vanish.* .. 

» 

The basic phenomena for factor analysis are the correlations of a set of 
variables. An explanation of these correlations can be made in the form of 
a factor solution in an infinite number of ways.t This indeterminacy is ex- 
hibited by the fact that various systems of factors may be selected, yielding 
solutions which reproduce the observed correlations equally well. In order 
to remove this indeterminacy, certain statistical criteria may be introduced. 
A number of such standards are given in Chapter V which lead to a few 
types of preferred factor solutions. 


2.7. B-Coefficients 


One of the fundamental problems in factor analysis is the selection of the 
descriptive variables. These should be chosen so as to measure the aspects 
of the group which are significant for a particular problem, i.e., the set of 
variables should be valid as a whole. This criterion implies that generally 
the intercorrelations will be positive throughout, or else can be put in this 
form (see 5.7). The variables should also be selected in such a way as to 
make possible the identification of the underlying factors. For this purpose 
it is necessary to choose at least three or four variables of a kind which ap- 
Pear to measure the same factor. The variables in each group of this type 
Should not be mere parallel forms but should be distinct in content. This 
hypothetical design of the variables is tested by the factor analysis, which 


* “The Velocity of Light," Scientific Monthly, 1939, p. 484. 


T For proof see Appen. В.З. It is shown there that this indeterminacy corresponds to 
the infinite number of rotations from one system of reference axes to another. 
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gives evidence for retaining or rejecting the original groupings of the vari- 
ables. 

In actual practice the above groupings are usually based on previous re- 
search in which some of the factors have already been identified. The design 
can then be extended to include other groups of variables used to identify 
additional factors. In some cases it may be desirable to take a portion of a 
previous design and add new variables to obtain more refined measures of 
the factors already identified. The success of the factorial analysis depends 
in a large measure on the skill with which the variables in such groups have 
been selected. 

If no design can be predetermined when the set of variables is com- 
piled and the matrix of correlations obtained, the grouping of the variables 
may be indicated by an objective procedure. This method is based on the 
assumption that the variables of a group identifying a factor should have 
higher intercorrelations than with other variables in the set. То this end, 
the B-coefficient, or coefficient of belonging, may be employed. This coeffi- 
cient is defined as 100 times the ratio of the average of the intercorrclations ofa 
subset or group of variables to their average correlation with all remaining 
variables. 'The group of variables for which B is computed is referred to as 
the "argument of B," while the remaining group of variables is called the 
"complementary subset." The number of variables in the argument of B 
increases, beginning with two, at successive Stages, and the value of the 
B-coefficient at each stage may vary accordingly. It is evident that the 
value of B for a given group of variables is independent of the order in 
which the variables are involved in the argument of B. On the other hand, 
the time when a particular variable enters into the argument of B is relevant 
to the value of B. For example, the B-coefficient of 21, 23, 24, Which is writ- 
ten B(z, 23, 24), or, more briefly, B(1, 3, 4), has the same value no matter 
what order the three variables assume. The value of B may differ, however, 
depending on the sequence in which another variable, say zo, is added. For 
concreteness, the value of B(1, 3, 2) is usually different from В(1, 3, 4, 2). 
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To define the B-coefficient more rigorously and thus obtain a formula for 
its computation, the following notation will be employed: 

n = total number of variables in a set, 

u = the subset of variables in the argument of B, 

р = number of variables in the subset и, 
j, k = indices ranging over и, 


cu = the complementary subset of variables, that is, those 
variables of the original set which are not in u, 


index ranging over cu, 


a 
ll 


Sa >, rj, = sum* of intercorrelations of variables in the 
(2.29) jck 
subset u, 


Т = ` rj, = sum} of correlations of variables in u with 
ч Jia 
those in cu, 


(3) = гази = number of combination of p elements 


taken two at atime;t here, the number of correlations 
involved in the sum 5, 


p(n — р) = number of correlations involved in the sum Т. 


* The symbol У means the sum of all the correlations r;; for the specified range 
j«k 

of j and k under the condition that j is always less than k. In the present section the 

Summation extends over all the p(p — 1)/2 combinations j < k of the p numbers taken 

two at a time. An illustration of such а sum in evaluating B(1, 2, 3, 4) may be written 


fully as follows: 
S = Ут = Tg + ris + ra + rd T TR. 


j«k 


; 1 An example of a sum Т for a total set of 6 variables in the evaluation of B(1, 2, 3, 4) 
ollows: 


T =J rja = Tis + rie + ree + Tos + ть + rs + Te + Tee. 
Жа 


In this example, n = 6, u = (1, 2, 3, 4), p = 4, j, k range over 1, 2, 3, 4, си = (5, 6), 
anda ranges over 5, 6. 


1 The symbol (") is used throughout the text in place of the conventional C? to desig- 
rn 


nate the number of combinations of n things taken т at a time. 
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Тһе B-coefficient of the subset of variables и is then defined by 


EUN 
(©) merus 


Gaza) "7 


For purposes of computation of B-coefficients it is perhaps more con- 
venient to write the formula for T in another form. Since the sums of the 
correlations of the respective variables with all others are usually obtained 


at the outset of any analysis, they may be used in the evaluation of Т for 
any B-coefficient. Thus, 


(2.31) d T= Dorie = Dorie — 29 ri = re- 28 
ja 


же j«k je 


(2.30) В(и) = 100 


„ 
n м 
where the index e ranges over the entire set of variables.* Тһе symbol 
$c stands for the sum of the correlations of the variables in u with all 
p 
‘other variables. Then to obtain Т it is necessary to subtract twice the inter- 
correlations of the variables in u. 

Another aid in the computation of the B-coefficients is the sum of the 
correlations of the last variable added to the group with the preceding ones. 


Letting l denote the last variable added to the group, the proposed sum 
may be written ы 


(2.32) L= У^, 


ЕРТІ 


where the summation extends over the variables in the subset м. If a sub- 
script p is appended to the sums L, S, and T to designate their values for 


* An illustration of formula (2.31) for the example of the last footnote follows: 


T =} ri - xa, 
іже j«k 
= (ти + nana + ris + rie) + (та + Tos F Tea + ras + та) 
+ (Ta + ra + Ta + ты + ra) + (ra + та + ra + ra + та) 
—2(ris + Tis F ra F Tos + Tru + та), 


= ть + Tie + Tos + ree + Tas + Tas H ras + тав. 


Although this formula for T seems to be more complex than that given in (2.29), its 
application to actual data is more effective than the latter. 
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p variables in the argument of B, then successive values of S and T may 
be obtained by means of the following recurrence formulas: 


(2.33) 6, = 8,1 + L, 
апа 
(2.34) Т = Шз + ra E 


еі 


where Yr is the sum of the correlations of l with all the other variables 
егі 
in the set. 

The value В(и) = 100 indicates that the variables of the subset u have 
the same average intercorrelation as their average correlation with all re- 
maining variables. Such variables would not be regarded as "belonging to- 
gether,” since they belong just as much with the other variables of the total 
set. As a tentative standard of belonging together, the B-coefficient of a 
group of variables should be at least 130. 

The B-coefficients are used to sort the variables on the basis of their inter- 
correlations. The grouping is begun by selecting the two variables which 
have the highest correlation. To these is added the variable for which the 
sum of the correlations with the preceding is highest. This process is con- 
tinued, always adding a variable which correlates highest with those already 
in the argument of B, until a sharp drop appears in the value of В. When 
this occurs, the last variable added is withdrawn from the group. Another 
variable may be inserted in its place, but, if the drop in B is still large, it 
should be withdrawn. Thus a group of variables that belong together is 
determined. Then, excluding the variables that have already been assigned 
to such a group, the two others which have the highest remaining correla- 
tion are selected to start another group. To these variables are added others, 
exclusive of those that have already been assigned to groups, until a signifi- 
cant drop appears in B, at which stage another group is formed. It is de- 
Sirable to start each new group with a B-coefficient as large as possible so 
ав to have clearly defined groups. For this purpose it may be necessary to 
obtain the B-coefficients for more than one pair of variables without com- 
pleting the groups. The pair yielding the highest B value may then be used 
to introduce the next group. This process is continued until all variables 
have been assigned to groups or else do not fit into any group. 

. At the present time по sampling error formula is available by which to 
Judge the significance of the difference between two successive values of В. 
One basis is to use all possible knowledge about the variables. It is a sound 
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scientific principle to use all the facts about the data of an experiment. The 
original hypothesis about the nature of the set of variables may thus aid in 
deciding whether a drop in B is "significantly" large. More objectively, 
some properties of the B-coefficient will now be presented. 

Since the B-coefficient is the ratio of two averages, its properties may be 
studied by means of them. Тһе average of the intercorrelations of the vari- 
ables и (the numerator of B) tends to decrease as the number of variables 
in B inereases, since the variables are added on the basis of highest correla- 
tion with those already in the argument of B. Similarly, the average of the 
correlations of the variables u with those in cu (the denominator of B) tends 
to decrease with an increase in p. The decrease in the numerator is relative- 
ly greater, however, than that in the denominator. 

To the numerator, which usually consists of a small number of correla- 
tions, are added a few additional correlations which are lower in value than 
the others and thus steadily decrease its value. On the other hand, from the 
large number of correlations in the denominator a small number of the larger 
values is taken away. The value of the denominator is decreased, but not 
so noticeably as that of the numerator. Thus the B-coefficient decreases, in 
general, as more variables are added to its argument. 

An exception to this may occur with the addition of a variable to the 
subset u which has relatively high correlations with the preceding variables 
but a low sum of correlations with those in cu. In this case the decrease in 
the numerator is relatively smaller than that in the denominator, and B in- 
creases. Similar reasoning accounts for the fact that a variable can be re- 
jected from a group because of a large drop in the value of В and then ap- 
pear in the group later, after several others have been added to the subset и. 

As the number of variables in the argument of B increases, the decreases 
in the above averages become less and these averages tend toward stability. 
A consequence of this is that an actual difference between two successive B 
values has a greater relative significance as the number of variables p in- 
creases. 

It is convenient to refer to the variables which are found to belong to- 
gether by the B-coefficients as constituting a group С, (s = 1, 2,..., т). 
Тһе serial numbers s may be assigned in order to the groups of variables 
which best measure the respective factors (other than a general factor). 
Thus, the variables at the head of the set that are found to belong together 
may be given the group number 1 and be said to belong to Gi. The factor 
which is predominantly measured by this group of variables may be sym- 
bolized by F;. Similarly for the other groups of variables, the final subset 
forms the group Gm, and the factor it measures is denoted Fm. If a solution 
involves a general factor Fo, the variables which measure it (i.e., all the 
variables) may be referred to as constituting a group Go. 


A 
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2.8. Illustrative Example 


In order to illustrate and clarify factorial methods, a numerical example 
is given here. This example is employed throughout the text whenever pos- 
sible. Various factor solutions will be obtained for the same data, and com- 
parisons made of different systems of factors. 


TABLE 2.1 


MEANS, STANDARD DEVIATIONS, AND 
RELIABILITY COEFFICIENTS 


Standard | Reliability 
ne Mean | Deviation | Coefficient 
T м gj un 
29.60 6.90 .756 
24.84 4.50 .568 
15.65 3.07 .544 
36.31 8.38 .922 
44.92 11.75 .808 
9.95 3.36 .651 
18.79 4.63 .754 
28.18 5.84 .680 
17.24 7.89 ‚870 
90.16 23.60 .952 
68.41 16.84 .412 
109.83 21.04 .937 
191.81 37.03 .889 
176.14 10.72 .648 
89.45 7.57 .507 
103.43 6.74 .600 
7.15 4.57 .725 
9.44 4.40 .610 
15.24 3.58 .569 
30.38 19.76: .649 
14.46 4.82 .784 
27.13 9.77 . 787 
18.82 9.35 .931 
25.83 4.70 ‚886 


The data employed for this purpose include a set of twenty-four psycho- 
logical variables (or tests), described briefly in Appendix B.1. In Table 2.1 
the means, standard deviations, and reliability coefficients are presented. 
These statistics are essential for the factorial methods that are treated. 

Тһе intercorrelations of these variables, which form the basic data for 
subsequent analyses, are presented in Table 2.2. Тһе complete set of cor- 
relations could be presented in the form of a symmetric matrix with the 
elements in the principal diagonal omitted. Obviously only half of this ma- 
trix is necessary for its complete description, and it is therefore presented in 
this form for simplicity. The correlations for any variable j are located by 
reading across the row j to the diagonal and then down the column j. Then 
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the sum of the twenty-three correlations of a variable j with all the others 
is given by the total of the entries in the row and column for test z;. This 


sum is represented by Уу , Where j is fixed and e ranges over the twenty- 
єў 

four variables under the restriction e + j. The sum of the correlations for 

each variable with all others is also given in Table 2.2. 


2.9. Application of B-Coefficients 


То illustrate the method of grouping variables outlined in 2.7, the ех- 
ample of 2.8 will be employed. The analysis into groups is begun by select- 
ing the two tests—5 and 9—which have the highest correlation, namely, 
Tss = .723. The value of B(5, 9) is computed by means of formula (2.30) in 
Table 2.3. The tests и, comprising the argument of B, are for this case 25 
and 29, and their correlation appears as the value of L and 8, since there is 
only this one correlation in each of the sums. The value T is given by 


T = Erse + Ero — 2rss = 8.242 + 8.156 — 2(.723) = 14.952, 
where the sums of the correlations are taken from Table 2.2. Then the 
value of the B-coefficient is 


. 200(n — p)S _ 4400(.723) _ 
"loai: our “mee 8, 


The form of computation indicated by Table 2.3 will be found very con- 
venient for machine calculation. In addition, the use of the sum L, defined 
in (2.32), and the recurrence formulas (2.33) and (2.34) will greatly facilitate 
the calculation of successive B values. 

To illustrate the use of these formulas, the value B (17, 18, 19, 15, 16) will 
be calculated, showing all details. Here, p = 5, and the last variable added, 
1, is 16. From definition (2.32) 


L= L; = УУ тыь = 117,16 + 718,16 + 719,16 + 715,16 = 1.251, 


J#16 


the individual correlations being given in Table 2.2. The sum S, or S;, may 
be obtained from the value S, by means of equation (2.33) as follows: 


8 = Ss = S4 + Ls = 2.001 + 1.251 = 3.252. 


It should be noted that there are two 54 values but that one has been ге- 
jected as indicated by note (10). The next entry is merely 


200(п — р) = 200(24 - 5) = 3,800. 


et 
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TABLE 2.3 
CALCULATION OF B-COEFFICIENTS 


| В(и)= | 

“ 1, S |200(п-р) D (p—1)T | 200(n—p)S | Notes 
| | 7-07 

.723  .723 4400 | 14.952 14.952. 918 ТТТ 
1.341 2.064 4200 | 20.179 40.358 215 |..... 

2.058 4.122 4000 | 24.245) 72.735 227 |..... j 

2.256, 6.378 3800 | 28.039) 112.156 216 |..... i 

2.276 8.654 3600 | 32.165 160.825; 194 (1) | 

( .585| .585| 4400 | 10.373| 10.373, 248 |..... ) 
( 920 1.5055 4200 | 16.117| 32.234 196 (2) 
(10,12, 11).... .912 1.497| 4200 | 15.638 31.276 201 (3) 
(10, 12, 11, 13) 1.455 2.952 4000 | 20.312) 60.936 194 (4) 
1.715 4.667 3800 | 25.102 100.408 177 (5) 
(10; 12, 11, 13, 21)..... 1.584 4.536 3800 | 24.750 99.000 174 (6) 


.509 .509 4400 | 15.415) 15.415, — 145 (7) 


.468 .468 4400 | 12.637| 12.637) 163 (8) 
.708 1.176) 4200 | 16.530) 33.060 149 
.865 2.041| 4000 | 19.551 58.653, 139 |... 
1.189 3.230 3800 | 24.770 99.116 124 (9) 


| ( 
-448 .448 4400 11.921) 11.921! 165 
.130, 4200 16.311| 32.622 145 


Ж -056, 4000 | 21.068 63.204 130 |(10) 
17,18,19,15)....... | .001| 4000 | 19.648 58.944 136 


1 
2 

(17,18,19,15,16).... 5 1.251 3.252 3800 | 23.755 95.000 130 К... 
6 


P 
2 
3 
4 
5 
6 
2 
3 
3 
4 
(10, 12, 11, 18, 24). ... i 
2 
2 
3 
4 
5 
2 
—— 
1,18,19, 16)... 4 


(17, 18, 19,15, 16,14). 6 1.530 4.782 3600 | 20.138 130.000 — 132 |... 
(618 3400 | 30.086 184.116, 122 | (11) 


icu sqm е 17 | 1.704 6.486, 3400 | 30.423) 182.538 121 (12) 
(20, 23)..... 42 .509  .509 4400 15.415 em MS loss 
(20, 23,22).... |8 .966 1.475) 4200 | 21.176) 42.352) 146 |... 
(20, 23, 22, 21). .... -| 4 | 1.238 2.713 4000 | 26.306 78.918, 198 (|4... 
(20, 23, 22, 21,24)....! 5 | 1.623 4.336! 3800 | 31.280| 125.120 189 мәре 
(20, 23, 22, 21, 24, 18). 6 1.652 5.988 3600 | 34.784 173.920 124 (13) 
(20, 23, 22,21,24,16). 6 | 1.641, 5 


кий | 3600 | 34.607 173.035) 124 (13) 


NOTES ON TABLE 2.3 
(1) Test 23 is rejected because of 22 points’ drop іп B for p=6. 
(2) Test 13 is rejected because 52 points' drop іп В seems to be too great even for p —3. 
(3) Test 11 is retained although it causes a drop of 47 points in B. Тһе drop іп B is not 
so significant when p —3 as it is when p is larger. Furthermore, Test 11 is of the same 
general nature as Tests 10 and 12. 
(4) Test 13 is retained, although it was previously rejected from the group. After Test 


11 was put in the group, Test 13 seemed to belong together with the others, causing 
a drop of only 7 points іп the value of B for p —4. 


(5) Test 24 is rejected because of 17 points' drop in B for р-5. 


BASIC STATISTICS 33 


(6) Test 21 is rejected because of 20 points' drop in B for p—5. 

(7) Before continuing with the group which starts with Tests 20 and 23, another pair 
of tests will be tried to see if they yield а value of B greater than 145. 

(8) Tests 1 and 4, although having a lower correlation than the pair 20 and 23, produce 
а higher value of the B-coefficient. Hence the next group is started with Tests 1 
and 4. 

(9) Test 21 is rejected because af 15 points' drop in B for p —5 and seemingly different 
nature of Test 21 from Tests 1, 2, 3, and 4. 

(10) Test 16 is rejected because of 15 points' drop in B to see if some other test will cause 
& smaller drop. If some other test cannot be found which causes a smaller drop in B, 
then Test 16 will be retained in the group at this stage because а drop of 15 points 
for p —4 does not seem to be definitely significant. 

(11) Test 24 is rejected because of 10 points’ drop in B for p=7. 

(12) Test 22 is rejected because of 11 points’ drop in B for p=7. 

(13) Tests 18 and 16, although they had previously been allocated to another group, are 
put into the argument of B along with 20, 21, 22, 23, 24 to see if the latter group 
of tests must be extended to other tests in the battery. The drop in B in each case, 
along with the seemingly different nature of Tests 18 and 16, seems to warrant their 
rejection from this group. 


The sum Т, or Ts, is given by formula (2.34) in terms of the preceding value 
T ,, which is in the row (17, 18, 19, 15) for p = 4, not in the row (17, 18, 19, 
16). Its value is 


T = T, = Tit Ут ав — 2L, = 19.648 + 6.609 — 2(1.251) = 23.755 , 


е”16 


where the sum of the correlations of Test 16 with all other tests is taken 
from Table 2.2. Then 

(p — DT = 4(23.755) = 95.020, 
and 


200(n — p)S _ 3800(3.252) _ |, 
(p—-D)T 950900 А 


B(17, 18, 19, 15, 16) = 


Following the procedure outlined in 2.7, all the variables are grouped by 
means of B-coefficients. The groups G, (s — 1, 2, 3, 4, 5), as determined in 
Table 2.3, may be defined by 


бл = (1,2,8,4), 
б, = (5, 6, 7, 8, 9), 

(2.35) Gs = (10, 11, 12, 13), 
Gs = (14, 15, 16, 17, 18, 19), 
Gs = (20, 21, 22, 23, 24). 
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Тһе grouping by B-coefficients adheres to the original design of this set 
of tests because most of these tests had been used in factor experiments 
before and because the abilities they measured were quite well known. This 
may not be true in general. In a number of published studies the B-coeffi- 
cients and the succeeding factor solutions failed to verify some postulated 
factors.* 


2.10. Correlations of Sums 


It is often convenient for subsequent analysis (e.g., Chaps. XI and XII) 
to consider a series of variables combined to form a single variable. Such 
new composite variables may then be put in standard form, and the correla- 
tion between two such variables obtained.| Thus it may be useful to com- 
bine the respective series of variables of each group б, (s = 1, 2,..., m), 
so that in place of the original n variables thére will be only m composites. 

Тһе reduction to composite variables and the intercorrelations between 
such derived variables may be done in the following manner. Consider two 
вегіев--2, Zz... , 2, and 2, Zu, . . . , 2o—consisting of д and Q variables, 
respectively. Each variable involves N values and is in standard form, i.e., 
has zero mean and unit standard deviation. Let the sums of these series of 
variables be denoted by the following composite variables: 


v=atat... +4, 
(2.36) Кик ва е4 


the values of which are obtained by substituting the corresponding values 
of the observed variables on the right. It is apparent that these new vari- 
ables are also measured from their means as origin. Since different letters 


* An example of this appeared in the factor solution obtained by Holzinger and Har- 
man, “Comparison of Two Factorial Analyses," Psychometrika, 1938, pp. 45-60. In pre- 
paring the test battery, Professor Thurstone had postulated that “verbal reasoning, 
numerical reasoning, and space reasoning would be separate factors and that, these would 
be different from verbal abstraction апа visual imagery" (see L. L. Thurstone, “Тһе 
Perceptual Factor,” Psychometrika, 1938, p. 11). Both his and the authors’ factor solu- 
tions cut across these predetermined groupings that had guided the test construction 
and revealed some different factors. 


1 Charles Spearman, “Correlations of Sums and Differences,” British Journal of Psy- 
chology, 1913, pp. 417-26. ы 


1 The variables zs, 21, . . . , Za should not Бе interpreted as denoting parallel forms of 
воше others. The roman subscripts are used here merely as а convenient way of desig- 


nating a second series of variables, distinct from the series 2, 2%,...,2,. 
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are used to denote the two composite variables, no subscripts are necessary 
to distinguish them, and a single subscript is sufficient to represent the in- 
dividual values of these variables. Thus, 
Ve = Abert... RR 
2.37 ig 
( ) un 


give the N values of the composite variables. 
Тһе correlation between the composite variables is given by 


Zvw; 
(2.38) Tow = Шыл? 


To obtain an expression for т„„ involving the correlations between the origi- 
nal variables, а formula for the standard deviation of a sum of variables is 
first required. By definition, 


_ 28 Z(nid- 2: d... + н)? 
gy — N N " 


Expanding the square, this expression becomes 


— „ [E X Ze Enni Eze, Zoi 
"S up ++ RE LEER 
Now, since 
2. VM 
(2.39) 5а =оф=1 and Dues = т, 


the last formula reduces to 


@ 
(2.40) в = |, +2 >, Т. 


j<k=1 


Similarly the standard deviation of the variable w may be written 


Q 
(2.407 ow = la +2 > ук. 
J«K-I 
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Returning to formula (2.38), it is seen that there remains Zv;w;/N to be 
expressed in terms of correlations between the original variables. Inserting 
the values of v; and w; into this formula, there results 


iE 


If the sums in the parentheses are expanded and the indicated multiplica- 
tion performed, this expression can be rearranged in the following form: 


4 9 
Zviwi 
(2.41) 5 = > Srv . 
j=l Jal 


Substituting the values from (2.40), (2.40’), and (2.41) into (2.38), the fol- 
lowing expression for the correlation between two sums of variables is ob- 
tained: 


(2.42) ee n M 


This formula gives the correlation between any two composite variables in 
terms of the correlations between the individual variables comprising the 
derived ones. 

As а special case of (2.42), the correlation of a variable zo, in standard 
form, with a composite variable v may be considered. It is readily seen that 
the correlation in this case is given by 


4 


> То; 


(2.43) йөз Ееее == 


|+» 


j<k=1 
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It may be instructive to give a diagrammatic representation of formula 
(2.42). Consider the following table of intercorrelations between the original 
variables involved in v and w: 


II Q 


Тап 


Ті 


Let Aq represent the sum of the elements іп the upper triangle, that is, 
the sum of the intercorrelations of the q variables making up the compos- 
ite variable v. Similarly, let AQ represent the sum of the elements in the 
lower triangle; and OQ those in the rectangle, that is, the sum of the in- 
tercorrelations of the variables involved in v with those in w. 'Then formula 
(2.42) may be written symbolically in the form 


же DR __ 
"C Vg + 2AqVQ + 24Q° 


(2.44) 


2.11. Application of Correlations of Sums 


To indicate the use of the formulas just developed, they will be applied 
to the groups of tests defined in (2.35). The composite variables arising 


from these groups are 


4 9 13 19 24 
(2.45) n= Уа, = > у, „= У2, п = > z; ъ= č; 
j=l j=5 


j=10 ісі 3-20 


where a single letter » with varying subscripts is employed to distinguish 


the variables. 
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The standard deviations of these new variables may be computed by 
means of (2.40), the result being 
c, = У4 + 2(2.041) = У 8.082 = 2.843, 


V5 + 2(6.378) = V17.756 = 4.214, 


Ov, = 
(2.46) ov, = V4 + 2(2.952) = У 9.904 = 3.147, 
oo, = V6 + 2(4.782) = V15.564 = 3.945, 
gu = м5 + 2(4.336) = У13.672 = 3.698 . 
ТАВІЕ 2.4 
INTERCORRELATIONS OF FIVE 
COMPOSITE TESTS 
Composit: 
et ө ті | т2 v3 T4 vs 
P ssaa ТҮ | sis pc арз lier ұн Әкелі 
PR. ее» ыды САВЕ Me m 
Uis. елин же, .334 39 
Ра мені Аз .425 427 « das 
Vis aya гы ‚594 | .649 | .545 | .588 


TABLE 2.5 


CORRELATIONS OF ORIGINAL WITH 
COMPOSITE TESTS 
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Then, applying formula (2.42), the intercorrelations of the composite tests 
are readily obtained. They are presented in Table 2.4, 

Besides the intercorrelations of composite tests, there will be occasion to 
use the correlations of the original twenty-four tests with the composite 
ones. Such correlations may be computed by means of formula (2.43). In 
Table 2.5 are given the correlations of each test z; (j — 1, 2,...,24) with 
the composite tests v, (s = 1, 2, 3, 4, 5) which do not include the test 2j. 


CHAPTER III 
GEOMETRIC FORMULATION OF THE FACTOR PROBLEM 


8.1. Introduction 


For a complete understanding of the various factor methods for analyzing 
а set of variables it is very helpful to resort to geometry to supplement and 
extend the algebraic ideas. The geometric foundation to be developed in 
this chapter will furnish a basis for subsequent analysis and comparison of 
methods. It will then be possible to use geometric ideas freely and also to 
pass from these to equivalent algebraic expressions. The geometry which is 
most useful for factor analysis is that of higher dimensions. The concept of 
extending the idea of dimensionality has been confined to the comparatively 
small class of mathematicians and philosophers, but psychologists and edu- 
cators have now come to rely on this concept as а very useful tool in analyz- 
ing mental traits, the total number of dimensions required being certainly 
greater than three. 

After à very brief exposition of the nature of higher dimensional geome- 
try, а coordinate system is introduced, so that the succeeding development 
сап be made analytically. Then the notion of linear dependence is devel- 
oped, which paves the way for one of the most fundamental theorems of fac- 
tor analysis. Before the application of these geometric ideas to the factor 
problem is made, however, certain formulas for distance and angle are nec- 
essary. These are developed, first, in terms of rectangular coordinates and, 
then, for general coordinates. The theory of transformations for rectangular 
coordinates forms the basis upon which some actual analyses are obtained 
in later chapters. The formulas in terms of general coordinates are included 
so that à geometric interpretation of less common forms of factorial solu- 
tions may be made. 

k In 3.7 a statistical variable is interpreted as a point, or a vector, in higher 
dimensional space. The standard deviation of the variable then becomes a 
distance, and the correlation between two variables is the cosine of the angle 
between the two vectors representing the variables. The direct application 
of the geometric theorems to the fundamental problems of factor analysis 
is made in the final section. There it is shown that the dimension of the 
smallest space which contains the vectors representing a given set of vari- 
ables is equal to the rank of the matrix of correlations with communalities 
in the diagonals. 
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3.2. Geometry of N Dimensions 


Тһе concept of higher dimensions is arrived at by geometric and alge- 
braie means. The notions of point, line, and plane may be generalized to 
higher dimensional objects, and extended geometric interpretations of al- 
gebraic relationships may be given. The deductions made in the higher 
dimensional spaces are based upon the analogous theory in three-dimension- 
al space and supplementary logical reasoning. 

The basic axioms* for Euclidean geometry may be assumed and such 
modifications made as are necessary to insure that the space has a sufficient- 
ly high dimensionality. The point, straight line, and plane are taken as 
undefined elements, and later corresponding elements of higher dimensional 
space may be defined in terms of these. 

Starting with four given non-coplanar points, all the points, lines, and 
planes can be obtained which constitute а three-dimensional space. The 
space, or manifold, determined by these points is essentially ordinary space 
of three dimensions. If the existence of at least one point not in this set is 
postulated, then all the elements of a four-dimensional space can be ob- 
tained (with the help of the basic axioms). The three-dimensional region 
does not now constitute the whole of space but merely a subspace of the 
space of four dimensions. The three-dimensional region will be called а 
hyperplane lying in the four-space, analogous to a plane lying in a three- 
space. A hyperplane in a space of four dimensions is determined by four 
non-coplanar points, a point and a plane, or by two skew lines. 

Some of the elementary geometrical properties of the elements in a three- 
and four-dimensional projective space may now be enumerated.] In a three- 
dimensional space two planes intersect in а line; a line cuts а plane in а 
point; and any three planes have a point in common. In a four-dimensional 
space two hyperplanes intersect in a plane, three hyperplanes intersect in a 
line, and four in a point, while five do not in general have any point in com- 
mon; a hyperplane cuts à plane in a line, and a line in a point; two planes 
have in general only one point in common, and a plane and a line in general 
have no point in common. 

The notion of dimensionality may be viewed in another manner. A point 
in a line is said to have one degree of freedom (of motion); in a plane, two; 
and in ordinary space, three. The point being taken as element, a line is said 
to be of one dimension; a plane, two; and ordinary space, three. These 

*D. M. Y. Sommerville, An Introduction to the Geometry of N Dimensions, 1929, 
Chap. I. See Appen. B.4 for list of basic axioms. 

+ By postulating a projective space the statement of the geometric properties is greatly 
simplified by not having to deal with special parallel instances. See Appen. B.4 for the 


projective axiom. 
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spaces are called linear spaces, or flat spaces, i.e., a plane is a two-flat and 
ordinary space is a three-flat. An (N — 1)-flat in an N-space will generally 
be called a hyperplane. The linear spaces point, line, plane, three-flat, . . 
hyperplane, N-flat are manifolds determined by one, two, three, four, .. . , 
М, N +1 points,* respectively, and having zero, one, two, three, ..., 
N — 1, N dimensions. 


3.3. Cartesian Coordinate System 


The geometric ideas are found to be most useful and easily formulated 
when they are given analytic representation. A point P may thus be герге- 
sented by a vector (zi %2,..., tw), which is a matrix with one row (or 
column). Each z; is a real number, and all N numbers may be called a sys- 
tem or N-tuple. By a '*point" is meant simply one of the undefined elements 
of the space which is characterized by а given set of axioms, so that a set 
of points is really an arbitrary set of any whatever elements. On the other 
hand, the N-tuple (zi, 22, . . . , zw) may be called an “arithmetic point." А 
correspondence between a set of geometric points" and a set of “arithmetic 
points” is called a coordinate system.1 The numbers, zi, 2o, ..., Жм, which 
constitute the representation of P, are called the coordinates of P. For 
purposes of factor analysis, the distinction between a "geometric" and the 
corresponding "arithmetic" point is not essential, and the word “point” will 
be used for either one. The notation P : (z;) will frequently be used to 
designate the point and its coordinates. 

An N-dimensional Euclidean space will be assumed, and in this space а 
non-homogeneous Cartesian coordinate system will be set up. The points 
О : (0, 0,...,0), and Еһ: (1, 0,...,0), Ea: (0, 1, 0,...,0),..., 
Ey : (0,..., 0, 1) will be called the origin and unit points, respectively. 
The N lines Oz; (i = 1, 2, . . . , №), each passing through the origin and 
one of the unit points, will be called the coordinate axes. The N hyper- 
planest т, = Ora... .)ai(... zw, each passing through О and containing 
N — 1 axes, will be called the coordinate hyperplanes. A hyperplane m; is 


* [t is understood that the set of p points, which determine a (p — 1)-flat, do not lie 
in & (p — 2)-flat. 


t The coordinate systems that will be introduced in this volume will always produce 
a one-to-one correspondence between the geometric and arithmetic points. This restric- 
tion may be removed, however. А correspondence may carry each point P into a set of 
arithmetic points, as, for example, in а homogeneous coordinate system. А theory of 
factor analysis based on such a system, or an even more general geometric foundation 
may prove to be very important, but thus far such research has not been undertaken. ' 


З К y 
і The inverted parentheses are used in the designation of any hyperplane to indicate 
the omitted coordinate axis. 
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said to be “opposite” to the axis Oz;. The coordinates (21, T2, . . . , тм) of 
any point P are equal, respectively, to its distances from each coordinate 
hyperplane measured along a line parallel to the opposite axis; or, in other 
words, the distance cut off on each axis by a hyperplane parallel to the re- 
spective opposite coordinate hyperplane. For exemple, the coordinate z; is 
equal to the distance (denoted by 21) cut off on the Oz, axis by a hyperplane 
parallel to the coordinate hyperplane ті. 


3.4. Linear Combination and Dependence 

The N-tuple (zi, To, . . . ; жу), Which represents a point P, may be con- 
sidered as a vector which joins the origin O to the point P. Such a vector is 
sometimes called a “radius vector." Two fundamental operations in vector 
algebra are multiplication by a number and addition of vectors. More pre- 
cisely, if P is a point represented by the vector (zi, t2, . . . , zy) and c isa 
number, then cP is the point 


(бал, 07), <.., CBN) ; 


while, if P, : (zu, Zi «+» tiv) and Р»: (Tan, 22, . . . ам) are two points,* 
then P; + P; is the point г 


(са + 22, Lig + Хз,...,Ш + том). 


In general, any linear combination of m points, Рі: (tu, 22, . . . , худ) 
T i 
o, Ры: (Emi, Төгу...) tuy), шау be defined by combining the two pre- 


vious operations, as follows: 


ҺР,+ЬР„+...+ҺР=, 


where the /в are any numbers. By taking varying values of the 26, different 
linear combinations of the original m points can be obtained. Any one of 
these new points may be denoted} P(t) or P(t, t, . . . , tm), with coordinates 
given by 


(3.1) ж = M unus G —1,2,..., N), 
8-1 


* The double subscript notation is used оп the coordinates in order to distinguish the 
points. Thus z,; designates the ith coordinate of the point Р,. 

f The symbol Р(0), or P(t, 0, . . - , tm), is the conventional function notation which 
is to be read, “Р is a function of 2 (in this case, a set of (в), ог P is a function of й, tz, 
22235,” On the other hand, P : (ә) is the notation for & point P with coordinates z;. 


44 FACTOR ANALYSIS 


and are said to be linearly dependent on the original points Pi, Ps, . . . , Ри: 
Each coordinate т; of a point P(t) is expressed as а linear combination of 
the corresponding coordinates zi; 221, . . . , Tmi Of the m points Pi, Рә,..., 
P,,.* A set of two or more points is said to be linearly independent if no 
one of them is dependent on the rest. According to these definitions, the 


origin is linearly dependent on any set of points and a single point is linearly 
independent if it is not the origin. 

The preceding definitions can be made, alternatively, by giving a direct 
definition of linear independence. Thus, a set of points Pi, . . . , Pm is linear- 
ly independent if the conditions 


(3.2) Хы = 0 
8-1 
imply that  -і-...-іһ-0. This is readily seen to be consistent 


with the definition (3.1). For if one of the coefficients were different from 
zero, say ћ >= 0, then (3.2) could be written in the form 


„В 
t 


t 
Эту — =" 


т = dy TT 4 


Tzi — Sod 5 


and, according to (3.1), the point P; would be one of the points P(t) which 
is linearly dependent on the points P», Ps,..., Pn. Having a positive 
definition of independence, the definition of linear dependence is given by its 
negation, that is, а set of m points is linearly dependent if the conditions 
(3.2) hold for the coefficients not all zero. 

When a set of points is given, it may be of interest to know how many of 
them are linearly independent. Let Рі: (zu, 2:5, . .. , тім), Р»: (221, 222, 
22.) Тәм),..., Р: (En Хн... Там) be any set of n points. Either all 


* To clarify some of these ideas, consider the special case of N = 3 and two points 
Py: (xu, 212, Хз) and Р»: (221, 222, тоз). All the points P(t) which are linearly dependent 
on the points P, and P: are given by the following coordinates: 


Tı = btn + bra 
P(t, te): 4 T2 = tit + т» 
Ta = litis + teres 


for varying values of tı and ё. For particular values of t, and t, the first coordinate of 
P(t) is a linear combination of the first coordinates of P; and Р;; the second coordinate 
is the same linear combination of the second ones; and the third coordinate is again the 
same linear combination of the third ones. For example, if the coordinates of P; are (1, 3, 
4) and those of P; аге (2, 1, 5) and t; = 1, t; = 2, then P(t) is given by the coordinates 
zı = 5,1% = 5, та = 14. 
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these points coincide with the origin or at least one of them, say P,, is in- 
dependent. Of the remaining points, either they will all depend upon P; or 
at least one of them, say Р», will be independent of Pi. Proceeding in this 
way an independent set of points, say Pi, Ps, .. . , Pm, will be obtained upon 
which all the points Pi, P2,..., Pa will be linearly dependent. A criterion 
for determining m may be obtained by means of the matrix 


Ті 113 ... Ty 

|Т 222 743 ... Tan 
Іі | = || £a Ти Tas ... Lay |, 

Хар Tne 243... Тау 


whose rows are the » points. Ап important result for linear dependence of 
points, and which will be utilized later to determine the number of common 
factors necessary to describe a sct of variables, may be stated as 

"THEOREM 3.1. If m is the rank of the matriz |x;;||, the points Py, Ps, . . . , 
P, are all dependent upon m of them, which are themselves independent.* 

Subspaces of the N-space may now be given analytical representation. 
If Py, Ps, ... , Px are k linearly independent points, the set of all points 
linearly dependent on them is called a linear k-space and is defined by the 
equations 


Р 
(8.3) zi = D ns (@@=1,2,...,N), 
іі 


where the /'s are a set of k parameters, and for each set of values (4,15,..., 
(4) there is a corresponding point of the linear k-space. Any one of the origi- 
nal k linearly independent points is, of course, given by definition (3.3); for 
example, Р» is given by в = 1 and & = =... = tj = 0. The k points 
Py, Py, ... , Px are said to determine the linear k-space. А linear 1-space 


* For a proof of this theorem see Appen. В.Б. An application may be made to the 
example of the last footnote. The matrix of the three points is 


13 4 
271 55 
5 5 14 


which is readily seen to be of rank two; for the third-order determinant is zero while а 
second-order determinant (e.g., the one in the upper left-hand corner) can be found which 
is different from zero. Since the matrix is of rank two, the three points are dependent 
upon two of them, which are themselves independent. Тһеп the three points are con- 


tained in a plane, but not in a line, as will be evident from Theorem 3.2. 
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consists of the points whose coordinates are proportional to those of a given 
point Рі: (zu, iy +- -3 там), and may be called a line through the origin. 
Its equations are given by 


(3.4) Ti = lm (= 1,2, ааа IM) 


These are a set of N parametric equafions of a line through the origin, where 
tı is known as the parameter. 

The transitive law for linear dependence may now be indicated. АП 
points linearly dependent on m points Py ..., Pm in a linear k-space are 
contained in that k-space. The coordinates of the m points are given by 
equations of the form (3.3), and any point linearly dependent on Pi, . . 
P,, is then obviously dependent on Pi, .. . , D 

Furthermore, if the points Рі,..., Р, determine a linear k-space, there 
is no other linear k-space containing these points. A linear k-space is thus 
determined by any set of k independent points contained in it, and а linear 
k-space does not contain а set of l independent points, where | > k. For, by 
definition (3.1), it is implied that any k points in a set of l independent 
points are themselves independent, and hence determine a linear k-space 
containing the larger set. Theorem 3.1 may then be stated as follows: 

THEOREM 3.2. If m is the rank of the matriz ||x;:\|, the points Ps, Py, . . . , 
P, are all contained in а linear m-space but not in a linear u-space, where 
и < т. 

А geometric interpretation of linear dependence сап now be given. The 
m vectors P, : (za, 22... , Zen), (S = 1,2, .. . , m), employed in the defi- 
nition (3.1), determine an m-dimensional subspace of the original N-space, 
and if OP;, ОР:,..., ОР, are taken as the coordinate axes, then ti, tz, 
... , min (3.1) are the coordinates of z;. 

A linear k-space, as defined by equations (3.3), always contains the origin, 
since the origin is linearly dependent on any set of points. The notion of 
subspaces of the N-dimensional space may be generalized to spaces which 
do not include the origin. For this purpose, a translation of coordinates, 


(3.5) Yi = Ti + с), 


is defined. Then any set of points which corresponds, under a translation, 
to a linear k-space may be called a flat k-space, or merely a k-flat. As noted 
in 3.2, a 0-flat is a single point; a 1-flat is a straight line; a 2-flat is a plane; 
and an (N — 1)-flat is a hyperplane. 

The geometric theorems developed in this section will be applied specifi- 
cally to the factor problem in 3.8. First, certain fundamental formulas for 
distance and angle will be developed and applied to correlational theory. 
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3.5. Elementary Formulas in. Rectangular Coordinates 

When the coordinate axes are mutually orthogonal, i.e., at right angles 
to one another, the reference system set up in section 3.3 is called a rectangular 
Cartesian system. Some elementary formulas in rectangular coordinates will 


be presented in this section. 
For any two vectors or points Рі: (zu, Тәу...) Tın) and Р»: (та, 2, 
‚ , жам), their scalar product* is defined by 


(3.6) P, + Po = Xn, 


where summation with respect to 7 is understood. Then the norm of P, is 
defined as the positive square root of the inner product of P, with itself, 


that is, 
(3.7) N(P) = УР; · Р; = У2а%;, 


and the distance between Р, and P» is defined by 
(3.8) D(P,Ps) = М(Р.- P2) = VZ(n — аа). 


It is readily seen that the norm of a point is the distance from the origin 
to the point, that is, N(P) = D(OP). The distance function satisfies the 
following familiar conditions of elementary geometry: 


DPP) = 0, 
3.9 D(PiP2) # O if Pi ғ P5, 
Ey D(P:P:) = D(PP) , 


D(P,P:) + D(P2P3) 2 D(PiPs) . 


The first three of these relations are obvious. The fourth, however, re- 
quires some proof. It may be noted that distances are invariant under 
translations. Thus if two points Pi, P; are translated into two points P1, 
Pi, then D(P,P;) = D(PiPj), which may be verified by applying (3.5). 
The fourth formula of (3.9) will therefore be unaltered if the points Pi, Ps, 
and P; are transformed by a translation which carries Рз into the origin. 
Then, putting in the distances by means of (3.7) and (3.8), the inequality of 
(3.9) becomes 


(3.10) VIr: + VER; E VZE(ns— ms, 


which may be verified algebraically. 


* This is sometimes called the inner or dot product. 
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Now the equality occurs in (3.10) if, and only if, 
їз = —lmu (4-1, 243% N), 


where t is а positive constant. These equations are of the form (3.4) and 
so represent & straight line through the origin with the points Рі and Рҙ оп 
opposite sides of the origin. Hence, equality occurs in the fourth relation 
of (3.9) if, and only if, the coordinates of P;, P», and Ps are related by equa- 
tions of the form 


(3.11) A(n; — 22) + В(ти — тыӊ) = 0, 


where A and B are constants of like sign and not both zero. If the condi- 
tion (3.11) is satisfied, and if P; # P» and Р» z Ра, then P; is said to lie 
between P, and Ps. 

Of special interest to factor analysis are some theorems of elementary 
geometry which have to do with transformations which leave distances 
invariant. Such transformations, in which any point Pi: (xi) is carried 
into Qi : (ji) and Р»: (та) is carried into Qs : (у), have the following 
property 


(3.12) (Жы = ai)? = Zu — ys? s 


From the condition that a point Р» is between two others, P; and Рз, if, 
and only if, 
D(P\P2) + (РАР) = D(P;P3) , 


it follows that a transformation which leaves distances unaltered carries 
straight lines into straight lines. Now, by a fundamental theorem of ge- 
ometry,* the transformation is linear, that is, of the form 


" 
(3.13) ун = DY onti б 
зі 


(@=1,2,...,N;j=1,2,3,...)- 


Upon substituting the values of yi; and ys; from (3.13), equation (3.12) be- 
comes 
N 


N N 2 
(3.14) ENG = tu) = 2, РЕ = Z 3 
k=1 


і-і i=l 


* The theorem states that any nonsingular transformation of an N-space into itself 18 
linear if it carries straight lines into straight lines. For a proof of this theorem see Oswald 
Veblen and J. H. C. Whitehead, The Foundations of Differential Geometry, 1932, рр. 
12-15. 


| 
| 
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It now remains to find the conditions which the a’s must satisfy in order 
that (3.14) should hold, and then the most general transformation which 
preserves distance will be specified. The right-hand side of (3.14) ean be 
written as follows: 

N 
E» [oed (vui — Ta)? + afa (ti — тә) +... + а (тау — Тау)! 
ізі 
T 2[a aas (T = Za)(riz - te) +... 
+ ai, yain (ni va = 23х21) (ту — тем), 


or simplified to 


N N N 
EN 
Ж >, M аата = Xx) (tu — ал). 


ізі К=1 ісі 
Hence, equation (3.14) is satisfied when 
(3.15) аай = би, 


where б is the Kronecker delta which is equal to unity if k = l and equal 
to zero if k = l. Any linear homogeneous transformation, 


(3.16) Yi = X ату, 


whose coefficients satisfy (3.15) is called orthogonal, and its matrix ап 
orthogonal matriz. 'The following theorem has thus been established: 

THEOREM 3.3. The distance between any two points is an invariant under 
a general rigid motion, that is, an orthogonal transformation followed by a 
translation. 

Other geometric ideas that are useful in factor analysis center around 
the notion of the angle between two lines. The only characteristic of a point 
is its position, as given by its coordinates in a frame of reference. А line is 
ordinarily distinguished not by coordinates but by its inclinations to the 
respective coordinate axes. The angles which a line OP makes with the 
axes, i.e., 0; = POz;, are called the direction angles of the line, and their 
cosines are called direction cosines. If the norm N(P), i.e., the distance 
D(OP), is denoted by p, then 


(3.17) zi = pcos 0; (@ == R EE: > 
By (3.7), 
р? = Еті, 
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and substituting the value of z; from (3.17), gives 


p = 5р? cos? h, 
so that 


(3.18) Z cos? 0; = 1. 


This property, that the sum of the squares of the direction cosines of a line 
in N-space is equal to unity, is a direct extension of the one in ordinary space. 

The parametric equations of a line through the origin O and a fixed 
point P, : (ты) are given by (3.4). The coordinates of any point P : (z;) 
on a line through the origin with the direction cosines 


№; = cos 0; 
are given by (3.17). When pistaken аза parametric variable along the line, 
the N equations (3.17) can be regarded as the equations of the line, which 
may be written 


‚= Le Боас dl 


Upon equating the М expressions for p, the following N — 1 equations 
arise: 


Tien ee = 201 
(3.19) md арда v 
If Р: (m, х,...‚, хм) is taken as a variable point on the line, (3.19) 


can be regarded as the equations of the line. 

By means of a translation, of the form (3.5), the equations of a line AP 
through an arbitrarily fixed point A : (а, az, . . . , ay) and with the direc- 
tion совіпев ^; are transformed from (3.19) to 


21-041 29 — 05 ім-а 
3.20 = un N N 
(3.20) € T m 


Moreover, if 
(3.21) м = bl; C= T9, әз 08 


where b is а constant different from zero, the equations of the line AP may 
be written in the form 


21-41 I» — а ZN-— а 
22 - = IERI 
(3.22) Т A FT js " 


CT Тт 


uro є" 
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where the 1; are not now equal to, but only proportional to, the direction 
cosines. The numbers l; are called direction numbers of the line. 

Тһе actual direction cosines of a line can readily be obtained from the 
numbers proportional to them. For, squaring both sides of (3.21) and sum- 
ming for т, this equation becomes 


where the last equality follows from (3.18). Then the constant of propor- 
tionality is 
1 


and the direction cosines are given by 


$e 


(3.23) i= Wee В 


Hence equations (3.22) may be taken as the general form of the (№ — 1) 
equations of a line in N-space. 

The coordinates of any point P : (т;) on a line through A : (а) with di- 
rection numbers l; are 


(3.24) z;— aid tl; (--1,2,.... Му, 


where 2 is the repeated value іп (3.22). Equations (3.24) may be regarded 
аз а system of parametric equations of a line through a fixed point. The 
distance D(A P) along the line from the fixed point А up to any position of 
the variable point P is 


D(AP) = VZ(z; — а) = tV, 
80 that, 


(3.25) t= Du 


М й 


It is thus evident that the parameter t in equations (3.24) is proportional 
to the distance from the fixed point to a variable point on the line and is 
equal to this distance when the equations of the line are given in terms of 
the direction cosines. 
Now a formula for the cosine of the angle between two lines in N-space 
may be derived. When two lines meet in a point,* a plane can be drawn 
* If the lines do not meet in a point, the angle between the lines may be defined as the 


angle which one of the lines makes with a line parallel to the second, which intersects 
the first line. 
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through the point containing the two lines, and their inclination can be 
obtained from the trigonometric properties of a triangle in the plane. Let 
the two lines through A : (a;) be represented by the equations 


mpg ie Ge. - Ту — ах 


№ № іы Ау ' 
(3.26) | 
и — Oi _ 10 — аз Yn — Ux 


ш Be МЕ ру 


, 


where the х; and y; are the coordinates of the variable points on the lines, 
and the A; and иу are the direction cosines of the lines. On the first line take 
any point P at a distance p from A; on the second line take any point Q 
at а distance 4 from А; and connect the points P and Q with a line, which 
necessarily lies in the plane. The points and lines are plotted in the plane 
of the two given lines in Figure 3.1. 


A:la;) q Quy) 
Fia. 3.1 


Let ф = angle PAQ and let d = D(PQ), then the law of cosines applied 
to the triangle PAQ gives 


(3.27) Ф = р + 4 — 2pq cos ф. 


The distance d is also given by formula (3.8), in which, according to (3.24), 


the coordinates of P are x; = а; + p^; and those of Q are у, = а; + qui 
so that 


[e = (жт — ys)? = Х(р№; — gus)’, 
(3.28) 4 = PEN + Pru — 9ра2Мш, 


| 


since EM = Sy? = 1 by (3.18). When the terms of (3.28) are identified 
with the corresponding ones of (3.27), the following result is obtained: 


pcd -—2pqXai, 


(3.29) cos ó = Eimi. 
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Thus the cosine of the angle of separation of two lines is given by the sum 
of the products of corresponding direction cosines of the lines, i.e., the scalar 
product of the vectors (А, №, ..., №) and (ui шу...) им). 

If the equations of the lines are given in the form (3.22), with l; and m; as 
the direction numbers of the respective lines, the cosine of the angle of 
separation becomes 


(3.30) cos ó = VE JE 


Here the cosine of the angle of separation of two lines is given in terms of 
numbers proportional to the direction cosines of the lines. 

By means of formula (3.29) another expression for the scalar product of 
two vectors or points may be written іс place of (3.6). The coordinates of 
the two points P; : (ты) and Р»: (те) may be expressed as follows: 


хы = pii, Las = рде; (521,2,...,N); 


where pi рг are the respective distances from the origin O to the points 
P3, Po, and Xii, Xy; are the direction cosines of the lines OP; and ОР». Then, 
substituting these values in (3.6), there arises 


Ру. Р» = тылы = ры Ан, 
which, according to (3.29), reduces to 
(8.31) Р.Р; = pip: COS фы, 


where фз is the angle Р.ОР». Formula (3.31) states that the scalar product 
of two vectors is the product of the lengths of the vectors by the cosine of 
their angular separation. This is very often taken as the definition of the 
scalar product. 


3.6. Elementary Formulas іп General Cartesian Coordinates 


In the preceding section various formulas were presented in terms of 
rectangular coordinates. Now the restriction that the coordinate axes are 
mutually orthogonal will be removed, and more general formulas obtained. 
The formulas for distance and angle will then be in terms of general Car- 
éesian coordinates and will simplify to those of 3.5 when the angles between 
all pairs of reference axes are taken as 90°. 

The general Cartesian coordinate system contains № reference axes Ох; 
which may make any angles with one another. Then the angle between the 
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х: and т, axes may be designated 0% (7, k = 1, 2,..., №). As may be 
expected, the formulas for distance and angle in terms of general Cartesian 
coordinates will involve the inclinations of the reference axes. 

Formulas for the distance function in general coordinates will first be 
given. In the plane the square of the length of the radius vector OP is 
readily found to be 


p = [D(OP)P = zi + z} — 2ту cos (180° — 0%) 
= 11 + 23 + 22120 cos 02. 


X, 


M:(x,, 0) 
Fig. 3.2 


This formula follows immediately on applying the law of cosines to the 
triangle POM, indicated in Figure 3.2. By induction, it can be shown that 


in N-space the distance p from the origin O to an arbitrary point P : (21, 
%,..., Zw) is given by 


(3.32) p = VZZziz,cos б, 


where ХХ indicates summation for 7 and k from 1 to N. This convention 
for the double summation will be employed throughout this section. In 8 
similar manner, the distance between any two points Рі: (тп, 
Там) and Ру: (n, Тә,.. 
formula: 


2105...) 
- Там) may be shown to be given by the following 


(3.33) D(PiP2) = VZZ(m; — 221) (Xie — Tox) COS б. 
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The relations (3.32) and (3.33) reduce to the corresponding formulas (3.7) 
and (8.8) for distance in rectangular coordinates when the angles of inclina- 
tion between pairs of axes are 90°, inasmuch as cos ĝi} = 0 for every 7 ¥ k. 

There will now be given several properties of & line in terms of more 
general (not necessarily rectangular) coordinates. The direction of a line 
OP is determined by the ratios of the coordinates of an arbitrary point 
P : (a, za, . . . , тм) on the line to the length o = D(OP). These ratios, 
denoted by ^; = z;/p, are called the direction ratios of the line OP. Then if 
the coordinates of P are expressed in the form 


(3.34) т; = pM ог хь = pr, 


and substituted in (3.32), there results 


р? = pEXAQN. COS Oik, 
so that 


(3.35) DEN; Ak cos 0; = 1. 


The direction ratios become the direction cosines of a line when a general 
Cartesian coordinate system is specialized to а rectangular one. Then for- 
mulas (3.34) and (3.35) reduce to (3.17) and (3.18), respectively. 

An expression for the angle between two lines, in general coordinates, 
can now be deduced. For simplicity, let the two lines pass through the ori- 
gin and be distinguished by the direction ratios А;, и; (2 = 1, 2,..., М), 
respectively. Select a point P : (z;) on the first line, and a point ©: (y;) 
on the second line, and let p = D(OP), q = D(0Q), d = D(PQ), and ¢ = 
angle POQ. Then 

@ = р + ф — 2pq cos ф. 


But d is also given by (3.33), in which the coordinates of P are z; = р\; 
and those of Q are y; = qu; so that after these values are substituted, the 
formula becomes 


Ф = р? + Ф — 2р@®®\М ик COS Ө. 


By equating the two expressions for the square of the distance, the following 
formula for the angle between two lines is obtained: 


(3.36) cos ф = ZZ; COS Oi. 


"This formula reduces to (3.29) when the axes make right angles with one 
another. 
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3.7. Geometric Interpretation of Correlation 
In this and the following section there will be presented a number of ap- 
plications of the preceding geometric ideas to the factor problem. Тіс raw 


data are the values of n statistical variables for each of N individuals. These 
data may be set down in a matrix as follows: 


| Xu Xe Ха Хау | 
| Ха Хь Xn Noy | 
Xil = | Ха Ха Хз X ||, 
Ds oom Hs & 41 
(Жа Xe Xa Xw | 


where any element X ;; represents the observed value of variable j for indi- 
vidual 2; so that the n rows represent the variables and the V columns, the 


individuals. By subtracting the mean of each variable from the observed 
values of that variable, the matrix of deviates, 


| £u 


Ti) Tiz ті | 
| Xo Ta T23 Ton | 
| 

m =f | 
12:1 = | n т» тіз Tay |, 
| 

| 

Tni Хз Таз ИҢ 


is obtained. 


Each row of | т) consists of телі, ordered numbers, and hence may 
be considered as the rectangular Cartesian coordinates of a point 2; (J = 
1,2,...,n). The variables (or points) are thus designated by an x with 
a single subscript, whereas the values of the variables (i.e., the coordinates 
of the points) are indicated by an т with two subscripts. A particular value 
(or coordinate) ту; stands for the ith coordinate of the point z;. 

As has been remarked before, it is usually more convenient to put the 


variables in standard form. The matrix of standardized values may then 
be represented by 


Zi 213 21у | 
| 2а 222 223 Zon | 
lza! = | 281 230 233 23у, 
| | 
Zni 2,2 213 Zan || 


in which the rows may be interpreted as the rectangular Cartesian coordi- 


nates of n points z; (j = 1, 2,..., n) in an N-space. 
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The length of the radius vector to a point z;, according to formula (3.7), is 


(3.37) pj = УХ}. 
According to equation (2.1), however, the expression on the right is equal 
to Уо, so that 


(3.38) р; = V/No; . 


The standard deviation of a statistical variable may thus be interpreted as 
being proportional to the distance from the origin out to the point represent- 
ing the variable, the constant of proportionality being 1/V/N. 

By way of geometric representation of a set of values of two variables, 
z; and ть, it is customary to think of the ту and ће т as the coordinates 
of N points (zi, хм), (тә, хв), ..., (Ziv, Ten) in the plane z;Oz;. This 
plot of points is called a scatter diagram; and, by means of this representa- 
tion, a better understanding of the relations involved in the definition (2.3) 
of a coefficient of correlation can be obtained. In general, for n variables 
this will be referred to as the point representation. 

Even more important in some respects is a geometric representation not 
by N points in a plane (for two variables) but by two points in an N-space. 
The two variables are then represented by the points x; : (zi, Zjs -> . , тім) 
and xy : (tm, Тәу...) Там), Which may be termed “vectors” according to 
3.4. Such a configuration for n variables will be called the vector representa- 
Lion. 

If the direction cosines of these vectors are denoted by dj; and А, re- 
spectively, then by (3.17), 


(3.39) Mim), M= — (-12...,N), 
2 


where p; = D(Oz;) and рь = D(Ox,). Inserting these values in formula 
(3.29), it becomes 
Утлы 


(8.40) соз фк = ZA = 
(2143 


where $;; is the angle of separation of the two lines. Then cos фу may be 
interpreted as the scalar product of the vectors z; and z, divided by the 
product of the lengths of these vectors. Upon substituting the values for 
р; апа p, from (3.38), formula (3.40) reduces to 


Хтуты 


(3.41) сов фк = 


Nojo, ' 
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which, according to (2.3’), becomes 
(3.42) Tik = COS ф;ь (= d, cuz g's 


The coefficient of correlation between two variables (measured as deviates from 
their respective means) is the cosine of the angle between their vectors in N-space. 


3.8. Fundamental Subspace for Factor Analysis 


By means of the geometric notions introduced in this chapter it is pos- 
sible to determine the minimum number of common factors that is neces- 
sary to describe a set of variables in the sense of equations (2.16). Ассога- 
ing to Theorem 3.2, the n points whose coordinates are given in the matrix 
|2,1 are all contained in a linear m-space, where m is the rank of the matrix. 
In other words, the л vectors can be described in terms of m reference vec- 
tors. 

This property can be stated in terms of the intercorrelations of the n 
variables by means of the following: 

Тнковем 3.4. The rank of the product of a matriz by its transpose 18 equal 


to the rank of the matrix.* 
The product of the matrix |12,1 by its transpose, |24||, is given by 


х2; 244и 744и ... Хг: 

Халы Ха; Хәй 2... ®ём2мы 
(3.43) 1:1 + 12:51 = | Жабы 22022: Ха)... Elani 

гыйы Хры: Хан ... Оғы 


Recalling that 22; = N and 52:26 = Nrj;, the preceding equation can be 
written in the form 


1 Та Та... Tin 


T21 1 Т?з... Топ 
(3.44) Ijzz] - lle] =N |та 732 1... tan |] =N [iridl- 
Tni Tn2 Tn3 .-.. 1 


Then, according to Theorem 3.4, the rank of the correlation matrix ||r;;|| is 
equal to the rank of the matrix of standardized values, ||z;;||. Hence any 


property of the variables which is inferred only from the rank of the latter 


* For a proof of this theorem see Appen. B.6. 
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matrix may be stated in terms of the correlation matrix. It therefore follows 
from Theorem 3.2 that the n variables can be expressed as linear functions of 
not less than m factors, where m is the rank of the correlation matriz. 

In case the correlation matrix contains ones in the diagonal, its rank is n, 
and the variables would then be describable in terms of n common fac- 
tors. If it is desired to describe the n variables in terms of less than n com- 
mon factors, a pattern of the form (2.16) may be postulated. From such a 
pattern the correlations are reproduced, as before, but with communalities 
in place of ones in the diagonals. Then a factor pattern of the desired form 
can be obtained by employing а matrix of the form in (3.44) with the ones 
replaced by communalities. The rank of this matrix is generally less than 
the order n. By the preceding argument, it is therefore apparent that the 
number of common factors in the pattern is equal to m, the rank of the cor- 
relation matrix. This is the smallest number of factors that will account 
for the intercorrelations. Stated geometrically, the smallest space contain- 
ing the n points is a flat m-space. Such a space will be referred to as the 
common-factor space. For purposes of reference the above ideas may be re- 
capitulated in the following: 

Тнковкм 3.5. If m is the rank of the correlation matrix, with communalities 
їп the diagonal, then the smallest number of linearly independent factors which 
will account for the intercorrelations is m; or, the common-factor space 18 of 
m dimensions.* 

In order to clarify the preceding ideas, the three important spaces will 
be reviewed. For any variable z; the system (гл, 22, ..., ziv) of N real 
numbers may be considered as the rectangular Cartesian coordinates of а 
point in an N-dimensional space. By means of this vector representation, 
the configuration of two variables is merely two dimensional, i.e., in a plane, 
although it has to be regarded as imbedded in an N-space. In general, the 
configuration of n vectors may be regarded as in an n-dimensional space 
which is imbedded in the original N-space. For purposes of factor analysis, 
this space can be greatly reduced, as indicated in Theorem 3.6. 

Before giving the final interpretation of the vectors representing the 
variables in the common-factor space, a geometric discussion of the linear 
expressions (2.4) which include unique as well as the common factors will 
be found convenient: The n vectors may then be considered in the total- 
factor space of the m common factors and т unique factors. The vector 
representation of any variable in this space is given by 


21: (ал, 5,5 єз ж Bim Ө; = 50,05 05.40), 


* This theorem is given by L. L. Thurstone, Vectors of Mind, 1935, p. 72. 
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where the prime is employed to indicate the linear description of the ob- 
served variable 2). The first m coordinates are with respect to the common- 
factor axes, and the last n coordinates, consisting of only one value different 
from zero, are with respect to the unique-factor axes. For simplicity let it 
be assumed that the common factors are mutually orthogonal, and, as 
usual, the unique factors are orthogonal to all factors. Then the norm, or 
length, of such a vector, according to (3.7), is 


(3.45) NG) = Vai +... +a, $a? = 1. 


In other words, each of the vectors representing the variables in the total 
factor space is of unit length. The direction cosines of such a vector in this 
space are simply the coordinates of the end point. The cosine of the angle 
of inclination (¢/,) of two such vectors, 2} and 2, then becomes 


m+n m 


(3.46) сов фу = 2. „М = Dd anan = тук, 


в-1 8-1 


denote the sets of direction cosines of 2; and 2), respec- 
hat the reproduced correlation for any two 
le between their vectors in the total-factor 
/, will approximate the observed corre- 
resentations of the variables are 


where M, and №, 
tively. Equation (3.46) shows t 
variables is the cosine of the ang 
space. The reproduced correlation 7, 
lation rj; to the extent that the linear rep 


adequate. 
Now the final interpretation of the variables as vectors in the common- 


factor space can be made. The orthogonal projections of the n vectors from 
the total-factor space into the common-factor space of m dimensions are 
defined to be the vectors representing the variables in this subspace. Such а 


vector may be denoted by 


zi' 1 (an, Qi, ++ , Ajm) - 


s of the end point of this vector are the same as the first m 
ce, This property holds even if the com- 
ded only that the unique-factor axes are 
be assumed for sim- 


The coordinate 
coordinates in the total-factor spa 
mon-factor axes are oblique, provi 
orthogonal to the common-factor space. It will again 
plicity that the common factors are uncorrelated. 

А projected vector in the m-space is usually of smaller magnitude than 
the corresponding vector in the total-factor space, being of the same length 
only if the variable has no unique variance. Likewise, the angles between 


A Сарла 


————M—— —— 
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pairs of vectors in the common-factor space are smaller and, consequently, 
their cosines larger. The length of a vector zj' in this space is given by 


(3.47) Ма)- Va t o... + Om = №, 


that is, the square root of the communality Aj of the variable. The direc- 
tion cosines of any two vectors 2)” and г,” in the common-factor space are 


given by 


(3.48) M TE Mp = 58 (s21,2,...,m). 


Putting these values into (3.29), the cosine of the angle of inclination of 
these vectors becomes 
Ж X ала 


DK a "nU LI LOS 
(3.49) cos $f, = > MAE =" 
It is obvious that the expression (3.49) is generally larger than that given 
by (3.46), being equal to it only when ҚА, = 1. Hence the angles between 
vectors in the common-factor space are smaller than the corresponding 


angles in the total-factor space. 
The problem of interpreting a reproduced correlation 7), geometrically 
can be treated in the common-factor space. It is evident from (3.46) and 


(3.49) that 
(3.50) COS Oj = FP: 
2 


Тһе совіпе of the angle of separation of two vectors representing variables 
in the common-factor space may be referred to as the correlation corrected 
for uniqueness. In other words, the expression (3.50) would be the value 
of the reproduced correlation between j and k if these variables were free 
from unique variance. Solving (3.50) explicitly for the reproduced correla- 
tion, there results 


(3.51) rj, = hh, cos фу. 


Thus, the reproduced correlation between two variables is given by the 
scalar product of their vectors in the common-factor space. Of course, the 
observed correlation r; differs slightly from the value given in (3.51), unless 
the residual is exactly zero. 
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where the prime is employed to indicate the linear description of the ob- 
Served variable z;. The first m coordinates are with respect to the common- 
factor axes, and the last n coordinates, consisting of only one value different 
from zero, are with respect to the unique-factor axes. For simplicity let it 
be assumed that the common factors are mutually orthogonal, and, as 
usual, the unique factors are orthogonal to all factors. Then the norm, or 
length, of such a vector, according to (3.7), is 


(3.45) Ма) = Май +... +a, +a? = 1. 


In other words, each of the vectors representing the variables in the total 
factor space is of unit length. The direction cosines of such a vector in this 
space are simply the coordinates of the end point. The cosine of the angle 
of inclination (¢/,) of two such vectors, z/ and 2), then becomes 


m+n т 
(3.46) cos Ф, = DN Me = Халы = т, 
8-1 8-1 


where №, and №, denote the sets of direction cosines of 2; and 2, respec- 
tively. Equation (3.46) shows that the reproduced correlation for any two 
variables is the cosine of the angle between their vectors in the total-factor 
space. The reproduced correlation r7, will approximate the observed corre- 
lation rj, to the extent that the linear representations of the variables are 
adequate. 

Now the final interpretation of the variables as vectors in the common- 
factor space can be made. The orthogonal projections of the n vectors from 
the total-factor space into the common-factor space of m dimensions are 
defined to be the vectors representing the variables in this subspace. Such a 
vector may be denoted by 


2j : (аў аљ... ат). 


The coordinates of the end point of this vector are the same as the first m 
coordinates in the total-factor space. This property holds even if the com- 
mon-factor axes are oblique, provided only that the unique-factor axes are 
orthogonal to the common-factor space. It will again be assumed for sim- 
plicity that the common factors are uncorrelated. 

A projected vector in the m-space is usually of smaller magnitude than 
the corresponding vector in the total-factor space, being of the same length 
only if the variable has no unique variance. Likewise, the angles between 
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pairs of vectors in the common-factor space are smaller and, consequently, 
their cosines larger. The length of a vector z;' in this space is given by 


(3.47) N(z/) = Уай + ae +... + Gin = №, 


that is, the square root of the communality A? of the variable. The direc- 

tion cosines of any two vectors z;' and z’ in the common-factor space are 

given by 

(3.48) Mi = 4% =” (s-12...,m). 
т 14 h; , hy , , , 

Putting these values into (3.29), the совіпе of the angle of inclination of 


these vectors becomes 
m 


nm > Gj ks 


" пұт с 3=1 
(3.49) cos $f, = > Nin = 5 а 
c= 
It is obvious that the expression (3.49) is generally larger than that given 
by (3.46), being equal to it only when Айх = 1. Hence the angles between 
vectors in the common-factor space are smaller than the corresponding 


angles in the total-factor space. 
Тһе problem of interpreting а reproduced correlation rj, geometrically 


can be treated in the common-factor space. It is evident from (3.46) and 
(3.49) that 


(3.50) сов фу = ТТ . 


Тһе cosine of the angle of separation of two vectors representing variables 
in the common-factor space may be referred to as the correlation corrected 
for uniqueness. In other words, the expression (3.50) would be the value 
of the reproduced correlation between j and k if these variables were free 
from unique variance. Solving (3.50) explicitly for the reproduced correla- 


tion, there results 
(3.51) rja = Қ соз 94. 


een two variables is given by the 
factor space. Of course, the 
51), unless 


Thus, the reproduced correlation betw 
scalar product of their vectors in the common- pace. 
observed correlation r;; differs slightly from the value given in (3. 


the residual is exactly zero. 
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А simple illustration of the foregoing ideas may be given for the case of 
only two factors. The common-factor space is of two dimensions, and the 
two (uncorrelated) factors F, and Р, are represented in Figure 3.3 by unit 
vectors separated by a right angle. Each variable z; of a set can be described 


кт 


22а, 022) 


н zia. а) 


Fie. 3.3 


іп terms of the two common factors and a unique factor. The linear ex- 
pressions for two such variables may be written as follows: 


(3.52) (4 = aul, + аЕ + 20 +0 - Us, 


г; = dnl’; + as Fa + 0 + Ui + а». 


The geometric representation of these linear expressions for the original 
variables can be made in the total-factor space of four dimensions, defined 
by the two common factors and the two unique factors. In this space the 
vectors representing 2; and 2; are of unit length, and their correlation is 
given by 


(3.53) Тір = апат *+ йй. 


АП the essential information about the variables, i.e., the intercorrela- 
tions, can be obtained from the consideration of the common-factor space. 
Тһе projections of the two vectors z; and 2; into this space are indicated in 
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Figure 3.3 by 21’ and zz', respectively, and may be written analytically in 

the form 
= ац F, 

(3.54) = а ans, 


x’ = а; + а»Ё» Y 


The lengths of these vectors are given by the square roots of their commu- 
nalities, i.e., 


(3.55) D(0s) = Уай + айз = УМ, 

5 ———- zs 
LD(0z/) = Val, + аһ = УМ. 

The cosine of the angle (9/2) separating these vectors is given by formula 

(3.49), as follows: 


ап ад, 42 0 _ 


1 
(3.56) cos $13 = he ti + ie, Quia (anan + ама»), 


or, employing (3.53), 
(3.57) тї, = ħħ, сов $13 - 


This formula shows that the reproduced correlation of two variables is 
given by the product of the lengths of the two vectors by the cosine of their 
angle of separation in the common-factor space. 

In the foregoing discussion it was necessary to employ distinct notation 
to clearly represent elements in the different spaces. Since it would be rather 
clumsy to retain primes and double primes in the remainder of this volume, 
they will be dropped when no confusion can arise as to the particular space 


involved. 


CHAPTER IV 
DETERMINATION OF THE COMMON-FACTOR SPACE 


4.1. Introduction 


In the preceding chapter it has been shown that the number of common 
factors necessary to describe a set of variables is equal to the rank of the 
correlation matrix. For a given set the observed correlations are known, 
but the communalities are unknown. The rank of the correlation matrix 
may then vary with the values put in the principal diagonal. It is thus pos- 
sible to determine values for the communalities which make the rank of the 
correlation matrix as low as possible. This produces a minimum number of 
common factors. 

Тһе minimum number of factors for a given matrix of linearly independ- 
ent correlations is obtained theoretically in 4.2. It is then shown that, when 
certain relations exist among the correlations,* the set of variables can be 
described in terms of a smaller number of factors. These relations are de- 
veloped explicitly for one factor and any number of variables in 4.3, and 
for two factors in 4.4. Тһе general procedure for obtaining such conditions 
among the correlations for any number of factors is indicated in 4.5. 

When the necessary conditions among the correlations of a set of vari- 
ables are satisfied for a certain rank of the matrix, the communalities can 
be evaluated consistently with this rank. This problem is considered in 
4.6 and illustrated with a set of eight physical variables. This method of 
determining the communalities forms the theoretical basis for the estima- 
tion of communalities diseussed in 7.2. Such estimates are required in the 
principal-factor and centroid solutions which are presented in Chapters VII 
and VIII. Finally, to illustrate some of the methods of this chapter, a ma- 
trix of correlations of six hypothetical variables is introduced. Even more 
important uses of this chapter, however, are made in the methods of analy- 


sis in Chapter VI. 


4.2. Number of Common Factors 
One of the major problems in factor analysis is to determine how much 
the rank of the correlation matrix can be reduced by a suitable choice of 


* When these relations exist, the property of linear independence does not hold. 
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the communalities when the intercorrelations are given. The correlation 
matrix may be written 


М Тө тз ... Tin 
Ті hi Та 2... Ton 
А = || ra 7m hi ex Fondly 
е 2 
fui Тө Ta ese M 


where the communalities, А2, are unknown, and the observed correlations 
satisfy the property, ri = ть; (J, Е = 1, 2,..., n). The rank of the sym- 
metric matriz А is m if ап m-rowed principal minor Ақт is not zero and 
if zero is the value of every principal minor obtained by annexing to Amm one 
row and the same column of А, and also of every principal minor obtained by 
annexing two rows and the same two columns.* By means of this theorem the 
number of linearly independent conditions that the unknown communalities 
must satisfy in order that the matrix A shall be of rank m can be determined. 

There are n rows in A and m in the nonvanishing principal minor Amm. 
This leaves (n — m) rows which may be annexed, one at a time, to Amm, ОГ 
(n — m) determinants which must vanish. The (n — m) rows may be added 


two at a time in (^ 2 "1 ways to Amm, giving (n — m)(n — m — 1)/2 
additional determinants which must vanish. Hence the total number of 
independent conditions (i.e., the number of minors set equal to zero) that 


the communalities must satisfy in order that А be of rank m ist 


(4.1) — ЧЁ (n — m = m-—1) _ (n — a m + 1) А 


In general, the vm equations have solutions A2, A2, . . - , А2 only if the num- 
ber of unknowns is greater than or equal to this number of conditions. If 
the number of unknowns is less than the number of equations, then the 
coefficients in the equations (the correlations) must satisfy certain relations 
in order that the number of independent conditions for the unknowns may 
be reduced to the number of unknowns. 


* L. E. Dickson, Modern Algebraic Theories, Theorem 15, p. 79. 


1 The proof given here is incomplete in the sense that the ул equations have not been 
Shown to be linearly independent, i.e., that none of them follows from the others. Walter 
Ledermann in his article, *On the Rank of the Reduced Correlational Matrix in Multiple- 
Factor Analysis" (Psychometrika, 1937, pp. 85-93), arrives at the same number of con- 
ditions for the n unknown communalities, although by 2 different argument, and offers 
a proof of the linear independence of these equations. 
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First, however, let it be assumed that the correlations are arbitrary 
values, then the set of conditions can be satisfied only if 


(4.2) Nn Eva. 


The last inequality may be written in the following equivalent form: 


. m? — (2n + 1)m t n(n — 1) 
2 


(4.3) om) = rm — т 6, 


ІЛ 


Setting the quadratic ¢(m) equal to zero and solving for m, the two roots 
are given by 


(4.4) т = 2+1 i У8п +1 


Now, it сап readily be shown that the plot of the quadratic function ф(т) 
is a parabola which opens up vertically. This curve crosses the m-axis at 
the two points whose abscissas are given in (4.4), and hence ф(т) € 0 for 
values of m between these extremes. The rank of the correlation matrix, 
with unknown communalities and arbitrary correlations, may thus be re- 
duced to the value m, which is given by 


(4.5) Gott Vet ome DED VERTI 


The smallest possible value for m is then the smallest integer greater than 
or equal to the value in the right-hand member of (4.5). In Table 4.1 there 
is listed the smallest rank that can be attained for a matrix of a given order, 
up to n — 15, when the correlations are assumed to be quite arbitrary. 


TABLE 4.1 


MINIMUM RANK UNDER ASSUMPTION OF 
INDEPENDENT CORRELATIONS 


23 45678 9 10 11 12 13 14 15 
m... | 1123324596 6 7 8 9 10 10 


з 


Generally, the observed correlations from statistical variables cannot be 
considered as arbitrary or independent. The inequality in (4.2) may then 
be reversed, that is, the number of unknowns may be less than the number 
of conditions which they must satisfy. The unknown communalities are 
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then "overdetermined" in the sense that the larger number of equations 
may not be consistent. In order for a solution to exist, the coefficients in the 
equations must satisfy at least ¢(m) relations. The differences (vm — n), 
that is, the number of conditions that the correlations must satisfy so that 
a matrix of order n can be reduced to rank m, are given in Table 4.2. 


TABLE 4.2 
NUMBER OF INDEPENDENT CONDITIONS ON THE CORRELATIONS: ф(т) 


N 2 3 4 516 7 8 9 10| 11) 12 n 

= —— 

1....|—1| of 2| 5| 9| 14| 20| 2735|44|54 (% қ E 1-58-90 
2...-2-2-Ц 1| 4| 8| 13| 1926/3443 (2) -2 n(n—5) | i 

3 -3|-3|-2 0| 3| 7| 12182533 қ” ? gut m ра 

4 -4-4-4-1 4 ena ("5 *)- 4 "а-9 16 

5 -s|-5|- 4-2) Ц 81016 (%%-5 n(n—10) | 19 

6 -4-4-8-4049(%9-6 nn—18) | 15 
с кушекн 


The lower left-hand corner of the table has no entries because the rank 
cannot exceed the order of а matrix. А negative value represents a larger 
number of unknowns than conditions, so that there is an infinite number of 
solutions in such a case, the general solution involving (n — vm) arbitrary 
parameters. À zero value represents the case of as many unknowns as equa- 
tions. For a given number of variables, n, a negative or zero entry indicates 
the rank m which the correlation matrix can attain without any restrictions 
оп the correlations. In these cases the inequality (4.2) is satisfied, and the 
conditions on the communalities are met under the assumption that the 
correlations are independent variables. The value of m for the first nega- 
tive or zero entry, reading down a column of Table 4.2, corresponds to the 


value of m, for the same n, in Table 4.1. 
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1% is important for factor analysis, however, to attain a lower rank, for 
any value of n, than that given in Table 4.1. In an actual factor study the 
investigator should select the variables on some hypothesis of underlying 
factors.* Then the correlation coefficients will not be independent, and, 
owing to the consequent relationships among them, the rank of the corre- 
lation matrix will be lower than otherwise. In other words, the number of 
factors, m, necessary to describe a set of п variables will be lower than the 
minimum value given in (4.5) if the variables are selected to fall into sub- 
groups. This, of course, is desirable in factor analysis. If ten factors were 
necessary to describe fifteen variables, there would be very little gain in 
parsimony of thought. 


4.8. Conditions for One Common Factor 


The number of independent relationships that must exist among the cor- 
relations in order that the rank shall be lower than the minimum in (4.5) is 
given by the positive values in Table 4.2. Thus for n = 3, no relationships 
are necessary to attain rank one; that is, three variables can always be de- 
scribed in terms of one common factor. Four variables, however, cannot be 
described by just one factor unless their intercorrelations satisfy two (inde- 
pendent) conditions. These well-known conditionst are the vanishing of 


the tetrads, namely, 


(4.6) 


Tiaa — Таз = 0, 
тала — Таба = 0. 


It тау be well to indicate how the conditions (4.6) are arrived at when 
n = 4 and m = 1, so that the method of generalization will be more evi- 
dent. When there are just four variables, the matrix of intercorrelations 
may be written 
hi Ті Тїз 714 
A. es Та h Ta Ta 
Ta 7з? hi T34 
Та Те Таз т 


This matrix will be of rank one if all second-order minors vanish. By the 
selection of appropriate minors, several linear equations can be determined 
for the solution of each of the communalities. Thus, hj is given by any one 
of the following three equations: 

№ та 


Tai 734 


hi Tis hi 714 


To Таз 


=0, 


Тәр 724 


* See 2.7. + Charles Spearman, The Abilities of Man, 1927. 
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or 


(4.7) № = 


Tus Тола Тізгіз 


T23 Tos T34 


On eliminating А, the two equations (4.6) arise. The solutions (4.7) for № 
are consistent if the conditions (4.6) are satisfied. 

Тһе question arises whether new conditions are introduced when the 
other communalities are solved for and eliminated. According to Table 4.2, 
only two independent conditions on the correlations are necessary to assure 
rank one for a fourth-order correlation matrix. To illustrate the dependence 
of any other conditions on the two already obtained, the solutions for the 


second communality аге given: 


(4.8) № = = = = 


Тіз Tis T34 


These equations may be obtained by setting appropriate second-order 
minors of A, equal to zero or, more simply, by interchanging the indices 1 


and 2 in (4.7). On eliminating Һ from (4.8), the following conditions arise: 
тала — Таба = 0, 
тігі — Таға = 0. 


(4.9) 


The first of these equations is identical with the second of equations (4.6), 
and the second is the difference of equations (4.6). Hence no new (independ- 


ent) conditions are introduced. : |! : 
Referring to Table 4.2 again, it is seen that five relationships must exist 


among the intercorrelations of five variables if they are to be described in 
terms of only one common factor. The correlation matrix, 
hi Тө Тїз 7 Ті 
Ta AR Tas Тм Ta 
As = || "з 732 № та 735 |] 5 
Та Taz Таз hi Ті 
hi 
Ты Тө Ті Гы 5 
will be of rank one if all second-order minors vanish. The five independent 
conditions that the correlations must satisfy may be obtained by the pre- 


ceding process. : 
Several linear equations for the solution of each of the communalities can 
be obtained by setting appropriate second-order minors of А equal to zero. 


Тһе first communality, for example. is given by any one of the six equations: 


hi Tia ET hi ШҮ жз 0, hi Ti5 =0 5 

Top T23 2 Тоу To4 Тор 7% 
№ тар M ns| o M ns! o 
Тз T34 mR Ta T35 ; Та 745 А 
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or 


Же Ті2Гіз 


Tii. _ Tiis — TisTi4 _ Tiaris 714715 
t = » 
Тоз T24 T25 T34 T35 Ta 
5 


(4.10) 


In order that the solution for А? shall be unique, the following five condi- 
tions must be satisfied: 


Ті7із ТізГі4 ТізГі5 ШЕШ Тіз” Tra? 
(4.11) = 5 14 13715 amis 


Тоз To4 T25 T34 T35 Та5 
These conditions may be put in the equivalent form: 


Tis o4 — Т47өз = 0, 
Tis es — Т?з = 0, 
(4.12) Tiaa — Т?з = 0, 
Tiaras — Ті = 0, 


Tiaras — 714735 = 0. 


Any other conditions must Бе linearly dependent on the above equations. 
Thus if instead of obtaining the solutions for hj, those for hj were obtained, 
the resulting five conditions could then be shown to be dependent on the 
foregoing relations.* А similar argument holds for the conditions obtained 
by solving and eliminating any other communality. 


* As in the case of л, there are six linear equations for the solution of М, namely, 


№ Tafa _ То1724 _ 721726 _ То3724 7237455 724725 
Тіз Tis Tis Tas Tas Tos | 


On eliminating h3, the consistency conditions may be put in the form: 


(i) таба — Таға = 0, 
(ii) Тат — Таба = 0, 
(iii) TuTs — Tats = 0, 
(iv) Tuas — Ты = 0, 
(v) Та — Tras = 0. 


Now it will be shown that these equations are linear combinations of those in (4.12). 
Equations (i) and (ii) are equivalent to (4.121) and 4.12;), respectively. Equation (iii) 
is the difference of (4.12,) and (4.123), and equation (iv) is the difference of (4.12;) and 
(4.12,). To show that (v) is linearly dependent on (4.12), substitute тіз = rier23/T24, 0b- 
tained from (4.12;), into (4.125). The result is 


туа а — 714725704 = 0, 


which reduces to (у) by factoring out 714. 


DETERMINATION OF THE COMMON-FACTOR SPACE 71 


According to Table 4.2, a set of n variables can be described іп terms of 
just one common factor if n(n — 3)/2 linearly independent relationships 
exist among the correlations. These conditions, whatever form they take, 
are equivalent to the following set: 


Tiris — Тїз _ Tiin  Тізгіі 
T23 T24 UC T2n T34 
(4.13) 
"S onus __ __ Тіп-171п 
Tan к Tn—ln ` 
For any variable ze, а term of the form 
"mcus eJ, дәй Oe uisum 
à Tjk exjx#k 


will be called a triad. It is readily seen that there are Е z '| triads іп 


(4.13), ог 


(ала) Г E ') кыз as 3) 


equations of condition for one general factor among n variables. 

Тһе number of conditions of the form (4.13) to determine whether a ma- 
trix is of rank one is considerably less than the number of tetrads. Every 
four variables give rise to three tetrads so that the total number of different 


tetrads for n variables is 


n n(n = 1)(n — 2)(n — 3) 
(и) зеле лев, 


(4.15) 


The difference between (4.15) and (4.14) is 


(4.16) [ez Э) . [se =) - J | 


То indieate the magnitude of this number, suppose n = 15. Тһе total 
number of tetrads is 4,095, while the number of triad conditions (4.13) 
is only 90. The difference (4.16) is 4,005. In other words, the labor of com- 
puting the conditions (4.13) is only about one-fortieth of that of computing 
the tetrads, for fifteen variables. For а larger number of variables the rela- 
tive economy of labor becomes more pronounced. 
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4.4. Conditions for Two Common Factors 


Тһе necessary conditions for a matrix of correlations to attain rank two 
will next be considered. For five variables it is a well-known fact that one 
relationship must exist, namely, the following pentad crilerion:* 


ТізГозГз47аб7ы — Ті9723735741754 — 11272473574351 + 710724731745753 
(4.17) H+ ryrestaarailss — Trl eslaiTaa7 sa — T1324 astar 52 1 718772577347 aol sl 


F ria eaT 317457502 — 7147257327 43761 — Т187293731742754 


| F твтатэотатьз = 0. 


This condition may be obtained by the method employed before. The cor- 
relation matrix, As, must have every third-order minor equal to zero if 
it is to be of rank two. By selecting appropriate minors, several linear 
equations for the solution of each of the communalities and hence the con- 
ditions for consistency can be obtained. Accordingly, for № the following 
two determinants are employed: 


hi Тїз Tis hi Tis Ті 
Tu Та 75| = 0, Ta Ta 7%| = 0. 
Tay Таз 745 T31 Тза T35 


Equating the two solutions for hj, there results 


та (тузт — таз) — Та(Тіз75 — Tis'o3) 
(73745 — Taslaa) 


Та(714735 — Tis 34) — 71(714725 — ШУЛ), 
(rearas — ТА?) 


, 


or 


[724735 = Tos? aa] [rei (713745 e ШИ, = 7а (713726 = Tis723)] 


(4.18) | 


== [resras = Tes? as] Гә1(714736 = тува) =< тз1(714726 = T15724)] = 0, 


as the single consistency condition. If the intercorrelations of five variables 
satisfy (4.18), or the equivalent condition (4.17), then five communalities 
can be determined to make the rank of A; equal to two, i.e., the five vari- 
ables can be described in terms of two common factors. 

According to Table 4.2, the correlations among six variables must satisfy 


* It is believed that this condition was first obtained by T. L. Kelley, Crossroads іп 
the Mind of Man, p. 58; see also Ledermann, op. cit. 
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four independent conditions in order for the matrix А6 to attain rank two. 
It will be convenient to make the following definition: 


hi Ті Та 
(4.19) |hi To Tea| = [Tao Tab Tan). 


Ter Ted Tea 


that is, a determinant is represented by the elements of the principal di- 


agonal. 
In order that Ав be of rank two, every third-order minor must vanish; 


five such determinants are 


|^i Тоз тав] = |n T23 Tsel = |ң Тоз ть] = |ң To4 ты| 


=|М rsa rss] -0. 


The solutions for А? from these five equations are as follows: 


(м Та1|7із Tas] = Ta|ris T25 
1 , 
| res 745 
A? = Ta | 713 тав] = Ta ris T26 
ht , 
| res 746 
Talr Tse| — "aria 7: 
(4.20) k = 1 13 561 ail 13 26 ; 
| ros 756 
№ = Т [714 тв | i Ты |714 To6 
2^ , 
| res Тв | 
№ = raltis Ты|- ты|ті 736 
бы , 
[r34 T56 
where 
Te Т, 
(4.21) |» Teal = |72 “| = тата — Тыт. 
Tob Ted 


Eliminating A? from the five equations (4.20), the four conditions which 
the correlations must satisfy are obtained. The equality of the right-hand 
members of equations (4.20) are then the necessary conditions for six vari- 
ables to be describable in terms of two common factors. 

This process can be continued for any number of variables. The posi- 
tive entries for т = 2 in Table 4.2 give the number of conditions that must 
exist among the correlations for a set of n variables to be describable in 
terms of only two common factors. In general, the correlations among n 
variables must satisfy (n? — 5n + 2)/2 conditions in order that their ma- 
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trix shall be of rank two. These conditions may be obtained by eliminating 
any communality А? from (n? — 5n + 4)/2 equations of the form 


та|Та Tah — Tee|Teb rad 
| rao Tea | 


БА” 


(4.22) k = 


еза з з сӯ й 


It will be noted that many more conditions than the number indi- 
cated in Table 4.2 can be written for the intercorrelations of n variables 
by means of the foregoing procedure. Corresponding to any four indices 
in the denominator of (4.22) there is a third-order determinant of the 
form (4.19) which is to be set equal to zero for the calculation of a particu- 
lar communality A2. Out of a given set of n variables there are i i VE es 
(n — 1)!/(n — 5)14! choices for the denominator, and each of these may 
be permuted in 4! ways. The total number of possible denominators, and 
hence, third-order determinants, for the calculation of the communalities 
would seem to be enormous. Fortunately, however, this number is con- 
siderably reduced owing to the symmetry of the correlation matrix and 
certain properties of second-order determinants. 

Although there are 24 possible permutations of four indices, leading to 
as many determinants (4.21), it will now be shown that these 24 determi- 
nants have only two different numerical values (and their differences and 
negatives). By means of definition (4.21) the following two properties are 


immediately verified: 
(4.23) [аЬ са) 
(4.24) jab cd| 


[ba «|, 
- [с ad|, 


M 


where, for simplicity, only the indices of the correlation coefficients are 
written. Then the two values of (4.21) for any four indices a, b, c, d arise 
from the successive applications of (4.23) and (4.24), as follows: 


(4.05) [аЬ cd| = |ба dc| = — |da be| = —|ad cb| = |cd abl 
= |de ба| = — {be da| = — |с5 ad], 

(4.26) [аЬ «| = |ba cd| = — |са bd| = — [ас db| = |dc ab| 
= |ed ba| = — |М са| = — |db ac|. 


Any one of the remaining eight determinants, resulting from another per- 
mutation of the four indices, is à combination of two expressions, one 
from (4.25) and the other from (4.26). For example, [ac bd| = |ab cd| 
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— |ab dc|, which may be checked by writing these determinants in full 
and making use of the symmetric property of correlations. 

Тһе evaluations of а communality (4.22) which are based upon the de- 
nominators indicated in (4.25) are all equal.* These equalities are quite in- 
dependent of the assumed rank of the correlation matrix. The investigator 
should employ thus only one of the equivalent denominators in (4.25). If 
he should employ all eight, and obtain eight identical evaluations of the 
communality, he would still have no assurance that the assumed rank of 
two for the correlation matrix is correct. The values of the communality 
employing the denominators in (4.26) are, again, all equal, but usually dif- 
ferent from the value involving a denominator of (4.25). If one of the re- 
maining eight permutations of four indices is employed, it can be shown that 
the resulting evaluation of the communality is а direct result of the pre- 
ceding two evaluations. 

To establish this property, consider the two evaluations of h? dependent 
upon the denominators of (4.25) and (4.26), respectively: 


N  ae|eb cd| — celeb ad | 


D [ab cd| á 
E N’ ae|e&b dc| — de|cb ас) 
өші =p" ГЕЛ 


Еш 


where E and E' are used to represent the particular evaluations, and only 
the indices of the correlation coefficients are written for simplicity. Consider 
also the evaluation employing |ac bd| in the denominator, namely, 


„_ N” _ае|ес bd| — be|ec ad| 
Е" = pa = [ac bd] 


* The equality of the eight evaluations of a particular communality employing the 
denominators of (4.25) is indicated in the following manner. Let E be the evaluation of 
а communality h? based upon the first determinant of (4.25), namely, 

_ Tae] reb rea]. — Tee |Teb Taal . 


£ |Tab Tea| 


The evaluation based upon the second determinant of (4.25) is 


тъ [теа тае] — Tae reo roc] 


x [roa таг | 


ть (Тете — TeTda) — тае(ГеаТье — Т«Ғы) 
= Thal de — Тыа 


та(ғыға — таты) — Te (Tiras — Тата) 
= $ 
Tabled — Tead 


which is evidently equal to Е. As one further illustration consider the evaluation based 
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It was indicated before that |ac bd| — |ab cd| — |ab dc|, that is, D" — 
D — D'. By expanding the determinants in the numerators, it follows that 
N = № = re(rara — Tara) — Те(таға — rara) 

— та (rarae — Таға) + та (rarae — TabTec) 
Tae (recta — ToeTea) — Ть(Те‹їва — Таға) 
ae|ec bd| — be|ec ad| = N”. 
Hence the evaluation E" can be expressed in terms of the other two evalu- 
ations as follows: 


\ 


N” М-М Ер-ЕП 
D D-DD D-D 


Е" 


If the evaluations E and Е” are reasonably close to the true value hj, then 
E” is also equal to this value, for, setting E = E' = hi, it follows that 
RD — D _ pp 


E DoD’ „Р 


The preceding discussion indicates that, of the twenty-four evalua- 
tions of a particular communality from the permutations of four vari- 


ables selected for the denominator of (4.22), only two determinations 


need be considered. For a set of n variables there are then 2 "n " 1. 


(n = 1)!/12(n — 5)! third-order determinants which must vanish in order 
that the rank of the correlation matrix shall be two. The number of condi- 
tions which arise upon equating the evaluations of a communality from 
these determinants is in excess of that indicated in Table 4.2 for n > 5. 
Although all these determinations of a particular communality must neces- 
sarily be equal (statistically) if the rank is two, these conditions are not all 
independent. The use of the large веб of conditions furnishes an exact check 
on the rank. In practice, however, it will be found sufficiently accurate to 
equate a smaller number of evaluations of communality for an approximate 
check of the rank, and for subsequent approximation of the communalities 
of a set of variables. An illustration of this is given in 4.6. 


upon the last arrangement of the indices a, b, c, d in (4.25), that is, 
E"- Tee|Teb Tad| — Toe|Teb Teal 
[Teo тоа | : 
which, on applying (4.24) to the denominator, becomes 
= [rae|res Tea] — Telres Taal] _ p, 
— |Tab rea] 


Е" = 


Similarly, any other evaluation employing the denominators of (4.25) can be shown to 


be equal to Ё. 
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4.5. Conditions for m Common Factors 


In the general case, the correlation matrix A will be of rank m if every 
(m + 1)-order minor vanishes. By the selection of appropriate minors, a 
number of linear equations can be obtained for the solution of each of the 
communalities. Then, by eliminating the communality, there results the 
conditions that the correlations must satisfy in order for A to be of rank m. 
A typical equation involving hj linearly is 


hi Тіз Т15 2.. Thaka өзе Тұз 
Toi T23 T25 <s.. 70,081 <s. Тоты 
Ta Таз 745 ааа Tabl e T4 m4 

D= =0. 
Торі Tana Taps +++ Тар, 0... T2j,2m41 
Tomi T2m,3 T2m,5 >... Тот,2к-1 + + + T2m,2m+41 


(j + 1)st row is given by the first index equal to 
the second index equal to 2k — 1. The determi- 
+ 1) columns. 

the elements of the 


It may be noted that the 
2j and the kth column by 
nant D is thus seen to contain (m + 1) rows and (m 

The determinant D may be expanded according to 
first column, as follows: 


МЮ = raDa +raDat --- (-іІ)т Ода... (-1)”т%»,Олһі = 0, 


where 
minor of the element hj, 


minor of the element rj; . 


Du 


The communality of the first variable is then given by 
(4.27) = (Cre Dann 
11 
s=1 


This solution involves the correlations among the first (2m + 1) variables, 
and, except for the first one, the correlations of an even-numbered variable 
are only with odd-numbered ones, and odd only with even. Other determi- 
nants than D, still employing only the first (2m + 1) variables, may be 
equated to zero to yield linear equations in hj only. Then different variables 
may be introduced from the original set of n to obtain still other linear 
equations in А. Thus the requisite number of equations of the form (4.27) 
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can be obtained, and from them, in turn, necessary restrictions on the corre- 
lations. The number of such independent conditions is given in Table 4.2. 
In the general case of л variables and m factors, the number of conditions 


is given by 


(n — m)(n — m 4- 1) _ n(n — 2m — 1) m 
уһ — n= 2 di 3 -(2). 


so that there should be one more than this number of equations of the form 
(4.27). Actually many more equations of this form can be written, just as 
in the case of rank two. Again, however, it may be remarked that a good 
approximation can be obtained by employing the indicated number of ex- 
pressions (4.27). 

In the preceding analysis it has been tacitly assumed that 


п > 2т +1, 


во that а determinant of the type D, involving the correlations among 
(2m + 1) variables, would be possible. If there were not this number of 
variables, then some other procedure would be necessary, for it would not 
be possible to obtain linear equations in only one communality. Perhaps 
the best procedure then would be to select (m + 1)-order determinants in- 
volving the smallest number of communalities to веб equal to zero. From 
a sufficient number of such equations the communalities could be eliminated 
to produce the necessary restrictions on the correlations. In a practical 
problem this difficulty is not to be expected because the variables are so 
selected as to form clusters or groups, and hence n is usually many times 
the size of m. 


4.6. Determination of Communality 


Up to this point various conditions for a correlation matrix to have а 
given rank were set down. These conditions were obtained by equating а 
number of evaluations of a particular diagonal entry or communality. It 
is then evident that this procedure could be employed in determining the 
communalites if the rank were known. Although the rank of a correlation 
matrix cannot be obtained directly, it may be approximated by the number 
of groups of variables, determined by such methods as those of Chapter II. 
The number of such groups is approximately equal to the number of linearly 
independent factors, and these in turn determine the dimensionality of the 
common-factor space. Then the converse of Theorem 3.5 states that the 
rank of the correlation matrix is equal to the number of dimensions of the 
space containing the vectors representing the variables. 
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In an actual analysis the number of common factors (or the rank of the 
correlation matrix) may be assumed to be equal to the number of groups G,. 
Then the methods of the preceding sections тау be applied to check 
whether the correlations satisfy the necessary conditions for their matrix 
to be of the assumed rank. If the correlations satisfy these conditions, then 
the communalities are given by equations of the form (4.22) for rank two, 
or, in general, by (4.27) for rank m. As will be indicated, the order of pro- 
cedure is to set down а number of evaluations of each communality for the 
assumed rank. The consistency of these values will serve as a check on the 
assumed rank, and their average may then be taken as the appropriate 
communality. Similar determinations would be required for each variable 
in the set. The assumed rank must check for all such determinations of 
the communalities. 

Of course, the correlations need not satisfy the conditions for a given 
rank exactly with actual data because allowance must be made for chance 
errors. À sampling error formula for the general expression in the right- 
hand member of (4.27) is not known at the present time. The standard error 
of the simplest instance of this expression (the triad), however, is developed 
in 6.5. When the rank of a matrix of correlations is assumed to be one, the 
true values of the various triads for the communality of any variable are 
equal. Thus by means of the standard error formula for triads, it is possible 
to determine whether the variations of the obtained values are attributable 
entirely to fluctuations in sampling. It is suggested that for rank one all 
Possible triads be written in the calculation of a particular communality. If 
the variation among these values can reasonably be assigned to chance fluc- 
tuations, the mean value may be taken as the communality. 

In the case of rank two, all possible expressions (4.22) for the determina- 
tion of each communality could be considered. Before being averaged, how- 
ever, those based upon insignificant denominators would need to be rejected. 
The variables yielding insignificant tetrads for the denominator of (4.22) 
can be identified when the design of the variables is known, and there are 
but two groups. In such a case each group of variables will approximate 
rank one, and the tetrads involving three variables of such a group will be 
insignificant. Knowing the combinations of variables which produce in- 
Significant denominators, it is not necessary to consider the expressions 
(4.22) which involve them. The denominators should include two variables 
of each group, considerably reducing the total number of expressions for 
each communality. 

When the rank of a correlation matrix is assumed, and the determination 
of the communalities is attempted, it may sometimes happen that some of 
the values exceed unity. Of course, such values of the communalities are 
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not permitted, and they indicate that the assumption of the rank is in- 
exact. Before the hypothesis of the specified rank is discarded, however, а 
number of evaluations of communality should be attempted. If, in general, 
several consistent values for each communality can be obtained, they should 
be averaged for the best determination of the communality, ignoring those 
values which exceed unity. The justification for this procedure lies in the 
fact that the observed correlations are themselves subject to error, and the 
values to be supplied in the diagonal of the correlation matrix to produce a 
specified rank can only be expected to satisfy this hypothesis approximately. 
'The final check lies in the agreement of the reproduced correlations, from 
the solution employing these communalities, with the observed correlations. 


TABLE 4.3 


INTERCORRELATIONS OF EIGHT PHYSICAL VARIABLES 
FOR 305 FIFTEEN-Y EAR-OLD GIRLS 


Variable 


. Length of forearm...... 
. Length of lower leg..... 
M'eIglibu: os un s vase кыл ae 
. Bitrochanteric diameter. 
. Chest girth .... soi irs es 
. Chest width........... 


If the final residuals are insignificant, then the choice of the communalities 
is statistically sound. 

Ап illustration of the calculation of the communalities when the design 
is known and the rank can be assumed to be two is furnished by the eight 
physical variables the correlations for which are given in Table 4.3.* The 
first four variables were selected as measures of 'lankiness" and the latter 
four as measures of “віосКіпевв.” This design is substantiated by the meth- 
od of B-coefficients. It is thus found that 


B(1, 2, 3, 4) = 235, 
B(5, 6, 7, 8) = 179. 


These values clearly indicate that the two sets of variables form two dis- 
tinct groups. Rank two can then be assumed and checked in the process of 
obtaining the communalities. 


* Frances Mullen, “Factors in the Growth of Girls Seven to Seventeen Years of Age” 
(Ph.D. dissertation, Department of Education, University of Chicago), 1939. 


mC —— 
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Assuming rank two, the communality of any variable z, can be obtained 
by averaging a number of evaluations of (4.22). Тһе calculation of such 
expressions can be facilitated by organizing the work as in Table 4.4, where 
the communality for the first variable in the given set is determined. 


TABLE 4.4 
CALCULATION OF hj 
ab cd ra|ns real — та [ть real N D № 
.846(.4622) — .473( — .0882) .4327 5474 7905 
.846(.4733) — .473(— .0422) .4204 .5379 .7815 
.846(.3612) — .473( — .0025) .3068 .3964 7739 
.846(.3733) — .398( — . 0422) .3326 .4253 .7821 
.846(.3426) — .398(— .0025) .2908 .3760 .7785 
.846(.3434) — .301( — .0025) .2913 .3765 7736 
.846(.4810) — .473( — .0487) .4300 .4873 .8823 
.846(.4958) — .473(— .0107) .4245 .4822 .8804 
.846(.3738) —.473( .0410) .2968 .3386 .8767 
.846(.4018) — .398( — .0107) ‚3442 ‚3904 - 8816 
.846(.3700) —.398( .0410) .2967 .3401 .8724 
.846(.3381) —.301( .0410) .2737 .3095 .8843 
.805(.4810) — .473( — .0448) .4084 4718 - 8665 
.805(.4958) — .473( — .0375) .4169 .4814 .8659 
.805(.3738) — .473( — .0096) .3054 .3534 .8643 
.805(.4018) — .398(- .0375) .3384 .3890 .8699 
КҮЗЕТІНДЕ .398( — .0096) 2.3017 8487 8651 
.805(.3381) — .301(— . 0096) .2751 .3189 .8625 
a E E E E PÁG libero Saray ais Ор ‚842 


In the stub of the table are recorded the four indices а, b, с, d which de- 
termine the denominator D of (4.22), namely, 


р = [Tab Tea | . 


These indices are selected so that two are included from each group, and 
duplications which arise by certain permutations, as indicated in (4.25) and 
(4.26), are omitted. Thus an orderly arrangement can be obtained by con- 
sidering all possible combinations of pairs of variables from the first group 
with every pair of variables from the second group. The variable for which 
the communality is being calculated must, of course, be omitted from such 
pairings. In Table 4.4 each pair of variables 23, 24, and 34 is considered 
along with each pair 56, 57, 58, 67, 68, and 78. Eighteen distinct values of 
the denominator, and hence the expression (4.22), are thus considered in 
Table 4.4. It is true that eighteen additional determinations could be made 
by interchanging the variables in only one of the pairs ab or cd. Since the 
computed values in Table 4.4 check the rank closely, it is highly improbable 
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that the additional eighteen determinations would be inconsistent with the 
former, and hence they need not be calculated. 

When the indices in the stub have been set down, it is suggested that the 
denominators in column D be computed. Some unnecessary work might be 
avoided by doing this first. For, if the original design of the variables was 
not accurate, three of the four variables might actually belong to one group 
(or measure a single factor), and the resulting D would then be very small. 
Such a combination of four variables would be rejected from the calcula- 
tions, and no further work would be done in that row of the table. 

After the denominators have been computed, cross out any that are in- 
significant, and then obtain the numerators of (4.22). For the communality 
of the first variable these numerators are 


М-та|ть Teal — тать ral. 


In the tabular arrangement of the work it is well to record all the coefficients 
та and тд before calculating the second-order determinants. It is also de- 
sirable to calculate all the second-order determinants of the first column, 
and then those in the second column, rather than the two in each row. With 
this procedure the successive second-order determinants can be calculated 
from Table 4.3 in an orderly fashion. By performing the indicated multipli- 
cations and combination of terms in each row, the numerators are obtained. 
These are then divided by the respective denominators to get the eighteen 
values of h?. The consistency of these values, as shown below, indicates that 
the assumption of rank two was justified. The mean is then taken as the 
value of the communality. 

It will be noted that many of the second-order determinants repeat them- 
selves in Table 4.4. This is true also of the determinants involved in the 
calculation of the communalities for the other variables, and many of the 
determinants involved in one table occur again in a later table for another 
communality. Hence the work involved in the determination of additional 
communalities, after the first, decreases. The complete calculations for each 
of the eight communalities will not be produced here, but the eighteen evalu- 
ations for each of the variables are given in Table 4.5. The assumed rank 
of two is checked by the consistency of the values in the columns of Table 
4.5. The mean values of the columns are then taken as the respective com- 
munalities. 

No exact standard for judging the consistency of these values is available, 
but & basis for determining the maximum allowable variation from the 
mean value can be set down. The calculated communality, although а de- 
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rived statistic, may be regarded as & variance, and the usual formula for 
the standard error of a variance applied to it. This formula is given by 


2 
(4.28) On = a А 


Аз in the сазе of other formulas involving derived statistics, it is expected 
that the values obtained from (4.28) will be smaller than those which would 
arise from a true formula developed for the communality. Therefore, if the 
variations of a set of values for а communality from their mean can be 
shown to be insignificant by use of formula (4.28), they would also be in- 


TABLE 4.5 


DETERMINATION ОЕ COMMUNALITIES FOR EIGHT 
PHYSICAL VARIABLES 


№ Aj м № hè hg № LH 


.7905 | .9301 | .8347 | .8332 | .9503 | .6117 | .5599 | .4871 
.7815 | .9279 | .8354 | .8416 | .9518 | .6105 | .5656 | .4813 
.7739 | .9258 | .8357 | .8378 | .9567 | .6063 | .5595 | .4793 
.7821 | .9270 | .8369 | .8386 | .9538 | .6093 | .5613 | .488U 
.1735 | .9261 | .8373 | .8388 | .9547 | .6075 | .5589 | .4842 
“7736 | .9255 | .8416 | .8378 | .9567 | .6065 | .5593 | .4808 
8823 | .8190 | .7534 | .8510 | .8297 | .7010 | .6360 | .4889 
18804 | .8147 | .7529 | .8561 | .8290 | .7023 | .6476 | .4766 
8767 | .8076 | .7513 | .8535 | .8268 | .7068 | .6367 | .4722 
8816 | .8133 | .7496 | .8541 | .8220 | .7075 | .6544 | .4798 
.8724 | .8138 | .7504 | .8551 | .8079 | .7224 | .6433 | .4759 
.8843 | .8080 | .7526 | .8519 | .8209 | .7116 | .6391 | .4735 
.8665 | .9090 | .8539 | .7552 | .8482 | .6234 | .5484 | .5475 
.8659 | .9028 | .8590 | .7608 | .8446 | .6224 | .5534 | .5401 
.8643 | .8945 | .8663 | .7489 | .8481 | .6247 | .5419 | .5408 
.8699 | .9060 | .8565 | .7556 | .8242 | .6127 | .5578 | .5431 
.8651 | .9036 | .8588 | .7500 | .8342 | .6287 | .5397 | .5463 
.8625 | .8962 | .8715 | .7480 | .8439 | .6262 | .5409 | .5435 


Co ity..... .842 | .881 | .817 | .815 | .872 | .647 | .584 | .502 
Gunter eds, ...{ .068 | .071 .066 | .066 | .071 .052 | .047 | .041 


Significant by the more accurate test. The standard errors for the eight 
communalities, as given by the above formula, are presented in Table 4.5. 
It will be observed that the maximum variation from the mean in any col- 
umn does not exceed 1.5 times the standard error, clearly demonstrating 
the consistency of these values and justifying the assumed rank and the de- 
terminations of the communalities. 

The preceding analysis is based upon а knowledge of the design of the 
variables in a given set. In case the variables are selected without a particu- 
lar design, they might still be grouped from the nature of their correlations, 
and the same procedure applied in the calculation of the communalities. If 
such grouping of the variables is not feasible, then the suggested selection 


of variables (e.g., as in Table 4.4) may be modified. 
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Without knowledge of the grouping of variables, which guided the selec- 
tion of the stub in Table 4.4, the indices might be selected on an arbitrary 
basis. One suggestion is to write а number of trials based upon variables 
selected with equal frequency from the whole set. An illustration of the 
stub of such a table for the caleulation of A? from a total set of eight vari- 
ables is presented in Table 4.6. The total number of distinct trials for the 
determination of any communality from a set of eight variables is 

(n—D!* , Т1 озду, 


12(n— 5)! 12(3!) 


ed on page 76. The proposed Table 4.6 includes only special 
namely, four consecutive ones in the first 
four with the first pair interchanged in 


as indicat 
combinations of four variables, 
seven trials and the corresponding 
the last seven trials. 
TABLE 4.6 
STUB FOR VARIABLES WITHOUT DESIGN 


Trial ab cd Trial ab cd 
la motae 23 45 8 32 45 
2 45 67 Dies e 54 67 
Dia куж ics Эй 67 82 10. 76 82 
4 82 34 ll. 28 34 
5 84 56 12. 43 56 
6 56 78 18: oes 65 78 
ы. к» мөл ҙе 78 23 М. oui un 87 23 


of two is verified by the evaluations from a set of 


e communality would be accepted 
hecking the rank, and computing 


If the assumed rank 
indices such as those in Table 4.6, then th 
as the average of these evaluations. In с 


the communality, the evaluations based upon insignificant denominators 
must be rejected. Thus the actual number of evaluations retained for check- 
han that 


nk and computing the communality may be smaller t 


ing the ra 
k on the rank could be ob- 


indicated in such a table. A more refined chec 
tained by employing additional evaluations. 
The direct application af the foregoing metho 
ties is practical when m is equal to one or two. For a larger number of f. 
the direct procedure becomes too cumbersome. Тһе methods of this 
asis of analysis into any number of common 


d of computing communali- 
ac- 


tors 
chapter, however, form the b: 


factors (see Chap. VI). 


DETERMINATION OF THE COMMON-FACTOR SPACE 85 


4.7. Illustration of Direct Solution 


For» small set of variables, when the communalities have been computed 
by the method of the preceding section, it may be possible to obtain a direct 
solution for the factor pattern. Such a solution is possible if, in addition to 
the knowledge of the communalities, a pattern plan is also postulated. This 
type of analysis will be illustrated with a set of six hypothetical variables. 

Assume a matrix of intercorrelations of the six variables as follows: 


79 75 49 42 28 


72 78 42 36 24 
75 48 35 30 .20 
А= 49 42 .35 42 28 
42 36 .30 42 24 


28 24 .20 .28 24 


Тһе problem is to formulate the pattern plan, determine the six communali- 
ties, and calculate the coefficients of the common factors. 

In order to compute the communalities, the approximate rank of A is 
required. This rank may be determined indirectly from the number of 
groups of variables. Applying the method of B-coefficients, it is found that 


BU, 2, 3) = 221, 
B(4, 5,6) = 92. 


ther while 4, 5, 6 do not 


The variables 1, 2, 3 are thus seen to belong toge 
hree variables. The fol- 


belong together any more than to the preceding t 
lowing groups are consistent with this analysis: 


Go = (1, 2, 8, 4, 5,6), 
Gi = (1, 2,3), 


and so the rank of A may be assumed to be two. А plausible factor pattern 
plan,* which takes these facts into account, is given in Table 4.7, where un- 
correlated factors are assumed for convenience. 

і Now the four necessary conditions which the correlations must satisfy 
in order that the six variables shall be describable in terms of two common 
factors are obtained by equating the right-hand members of equations (4.20). 


* The pattern plan is plausible because the correlations among the variables 1, 2, 3 
are higher than those among 4, 5, 6. Such a plan is consistent with the hypothesis that 
ап extra factor should be postulated for а group of variables with high intercorrelations. 
This hypothesis is implicit in all current methods of factor analysis. 
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The value of each of the first three of these expressions is exactly .74, while 
the last two expressions are indeterminate.* The correlations satisfy the 
necessary conditions, and hence А is 5f rank two. The communalities are 
then given by equation (4.22), for n == 6, and the average of the various 
determinations of each communality may be taken. For the present hypo- 
thetical problem the different solutions for any communality yield the same 
value. The computed communalities are presented in Table 4.8. 


TABLE 4.7 


FACTOR PATTERN PLAN^ 


* A factor pattern will usunlly be presented 
in such a tabular form with the cocflicients of 
the respective factors appearing in tlie columns 
headed by the factors. 


TABLE 4.8 
COMMUNALITIES OF Six HYPOTHETICAL VARIABLES 


From the intercorrelations and communalities, ай ће coefficients of the 
factor pattern can be obtained. Since the last three variables involve only 
one common factor, each of their communalities is merely the square of the 
coefficient of this factor. Hence 


№ = 49 = а, 
or 


Similarly, 
ав = 6 and йв = Ж. 


* This is due to the fact that the tetrads formed from variables 4, 5, 6 and any one of 
1, 2, or 3 vanish. As & consequence the minors in the numerators of these expressions 
also vanish. It is thus apparent that four such variables are describable in terms of only 


one common factor. 
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The coefficient a; may be obtained by means of any one of the correla- 
tions r/,, rl, or rfe; for, by (2.22), 


Tik = 404% (Б m4, 5, 6)» 


A similar argument holds for the determination of азо and aso. Then, em- 
ploying the mean of the three evaluations of any one of these coefficients, 


they are given by 
6 
1 Тік А 
а РЭГ (0= 1,2,3), 


where the correlations from the factor pattern (the 7),) are replaced by 
the observed correlations in А. The tacit assumption is that the residuals 
are zero. The coefficients of Ғ for the first three variables have the follow- 
ing values: 

is mm, ао = .6, азо = .5. 


Since the coefficient ало is known, the remaining coefficient an in the first 


equation of the pattern may be determined from Aj. For 
№ = а, + а, 


ог 
an = УВ — а = УЛА — 49 = .5. 


Similarly, ад = .6 and aa = 18. The factor pattern thus computed from 


the correlations in А is given in Table 4.9. 


TABLE 4.9 


FACTOR PATTERN FOR SIX 
HYPOTHETICAL 
VARIABLES 


Variable | Fo Fi 
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It may readily be checked that the correlations reproduced from the 
pattern of Table 4.9 agree exactly with those given in A. Thus, 


ті, = .1(.6) + .5(.6) = A2 = Tg. 


In ordinary practice, of course, some discrepancies between the original and 
reproduced correlations would be expected. The approximate agreement be- 
tween these values is the essential requisite for the adoption of any factor 
solution, the form selected being somewhat arbitrary. This is illustrated in 
the present example by postulating the plan of solution of Table 4.7. A dis- 
cussion of various preferred pattern plans which may be employed for а 
given set of data will be taken up in the following chapter. 


CHAPTER V 
PREFERRED TYPES OF ORTHOGONAL SOLUTIONS 


b.1. Introduction 

In the choice of a scientific hypothesis two possibilities immediately arise 
from the fact that the factors may be taken as correlated or uncorrelated. 
This leads to two distinct developments of the theory. In this and the next 
five chapters the common factors will be assumed to be uncorrelated, or geo- 
metrically the axes representing them will be taken as orthogonal. Then in 
Chapter XI the case of correlated, or oblique, factors will be discussed. 

When it has been decided to describe a set of variables in terms of un- 
correlated factors, the problem is not yet unique. There are still an infinite 
number of orthogonal reference systems in terms of which the variables may 
be described. The next problem is to select one of these frames of refer- 
ence, on some rational basis, as & preferred system. Certain preferred types 
of orthogonal factor patterns will be discussed in this chapter, after a con- 
sideration of some basic standards for judging the satisfactoriness of form 
as a simple scientific description of the variables. 

The extent to which each of these types of solution conforms to the basic 
statistical criteria will be discussed in detail. It is shown that by the intro- 
duction or elimination of one or more of these standards it is possible to pass 
from one preferred form to another. Thus it is hoped that some unification 
of factorial methods will be achieved. 

In this chapter only schematic, or formal, solutions of the preferred types 
are presented. Although the algebraic equations of the various factor pat- 
terns are exhibited, the formulas for the computation of the coefficients are 
not given here. The special analytical methods employed in obtaining nu- 
merical solutions of the preferred types will be developed and illustrated in 


detail in later chapters. 


5.2. Some Standards for Judging Preferred Patterns 

In order to limit the infinitude of possible factor solutions that can be ob- 
tained in describing a given matrix of correlations, a set of restrictions must 
be imposed to obtain some preferred reference system. To this end, a list 
of statistical standards is presented which is based partially upon those 
found useful in previous investigations. Some of these standards are analy- 
tical in character, while others are of a geometric nature. Inasmuch as both 

89 
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these types of standards are designed to produce simple forms of solution, 
certain of these standards are somewhat related. They are listed here sepa- 
rately, however, because they support one another and simplify the selec- 
tion of preferred solutions. By presenting such a broad list of criteria, the 
assumptions underlying each type of preferred solution can be exhibited 
explicitly. 


1. AGREEMENT WITH ASSUMED COMPOSITION OF VARIABLES 

The composition of variables postulated in Chapter II was based on 
three types of observation, namely, that correlations are found among vari- 
ables of a set, that potential linkages may also occur among variables in the 
set with others not included, and that all measurement is subject to error. 
These considerations led to the fundamental equations (2.8) involving com- 
mon, specific, and unreliable factors. All forms of solution should obviously 
conform to the factorial composition of the statistical variables postulated in 
such linear equations. 

2. PARSIMONY 

According to the principle of parsimony common to all branches of sci- 
ence, a law or description should be simpler than the data upon which it is 
based. This may be illustrated in the fitting of a theoretical curve to a series 
of observations. The number of constants in such а function should be much 
smaller than the number of observations in order to give a simple and useful 
interpretation of the latter. Similarly in a factor problem the functional de-_ 
scription of the variables should be much simpler than the original data. 

a) Number of common factors.—In agreement with this principle the total 
number of common factors should be considerably smaller than the number 


of descriptive variables comprising the original set. 
b) Complezity.—1t is also desirable that this principle of parsimony be 
applied to individual variables. In the linear description of each variable the 


complexity should be as small as possible. 


3. UNCORRELATED FACTORS 

Тһе first stage in any factorial analysis must involve а choice of correlat- 
ed or uncorrelated factors. In the assumed composition of variables there is 
no restriction as to whether or not the factors shall be correlated. The ob- 
served correlations are fitted equally well by solutions involving factors of 
either type. Because of the indeterminacy of factorial solutions, investiga- 
tors have found it convenient to start with an orthogonal solution. This 
form may then be retained or else rotated to some preferred orthogonal or 


oblique solution. | 
The advantages of а solution involving uncorrelated factors arise from 
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convenience of initial solution and subsequent interpretation of factors. As 
just indieated, an orthogonal solution is the fundamental form from which 
others may be derived. In some instances, however, such a solution may 
conform to a preferred type without further transformation. If uncorrelated 
factors are selected for the final solution, there is a decided clarity of in- 
terpretation, especially in the description of individuals in terms of factors. 
Such descriptions are clearer and more economical than if expressed in terms 
of interrelated factors. 
4. RELATIVE CONTRIBUTIONS OF FACTORS 


Another standard which may be useful in the selection of a particular 
type of solution is based upon the relative contributions of factors. As de- 
fined in 2.3, the contribution of a factor is given by the sum of the squares 
of its coefficients for all the variables. Three useful types of relationships 
between the contributions of a set of factors follow: 

a) Decreasing contributions.—In this case the various factors contribute 
successively smaller amounts to the total communality. 

b) Level contributions.—A second choice is one in which each factor con- 
ately an equal amount to the total communality. 

с) One large and remaining level contributions.—The third type of rela- 
tionship is that in which one factor contributes a large amount, while the 
tribute a much less but fairly uniform amount to the 


tributes approxim 


remaining factors con 


total communality. 
It can be shown that the contribution of a factor is related to its statisti- 


cal significance. This has been demonstrated both by theoretical treatment 
and by empirical evidence. 
5. GEOMETRIC FIT: VECTOR REPRESENTATION 


As pointed out in Chapter III, the geometric interpretation of the factor 
problem frequently adds clarity to the analytical method. This is especially 
true in formulating a set of standards leading to several distinct types of 
preferred patterns. When such distinctions have been made in geometric 
terms, the corresponding analytical properties of the factor patterns are 
explained. The following geometric criteria furnished the bases for several 
scientifically desirable factor patterns. 

a) Linear fil.—An obvious justification for postulating common factors 
is that the variables in a particular investigation usually are correlated. 
Furthermore, certain subsets of variables may show generally higher inter- 
correlations among themselves than with the remaining variables of the 
total set. Group factors corresponding to each cluster of this type may then 
be assumed. In a sense, then, a group factor may be regarded as a sort of 
average or the common element of the variables of such a subset. 
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It has been shown in 3.7 and 3.8 that the variables may be regarded as 
vectors with а common origin and that the correlations between variables 
are given by the cosines of the angles between such vectors in the N-space, 
or by the scalar products of the projected vectors in the common-factor 
space. A group of variables yielding a cluster of high intercorrelations is 
thus encompassed by a “сопе” with a relatively small generating angle. If 
a reference axis or vector of the common-factor space is chosen in the midst 
of this cone, all variables in the group will correlate high with it. To a fac- 
tor common to a group of variables, there thus corresponds a reference vec- 
tor. The degree of linear fit is measured by the compactness of the “сопе” 
representing the particular group of variables, or the extent to which the 
vectors of these variables approach the axis of reference. By selecting a 
number of such reference axes, each one passing through a cone of vectors, 
the whole configuration may finally be well fitted. 

The standard of linear fit, together with that of uncorrelated factors, 
usually can be met only roughly in the case of positive correlations among 
the variables. It is evident that a closer linear fit can be obtained by per- 
mitting the factors to be correlated, as will be shown in Chapter XI. 

b) Planar fit.—The type of geometric fit just described may also be in- 
terpreted as a planar fit. Each plane is defined by two of the reference axes, 
or by the end points of the two reference vectors and the origin. Good geo- 
metric fit is then indicated by the proximity of the vectors representing the 
variables in the common-factor space to such planes. 

A subset of variables may be well represented by vectors in а reference 
plane of the common-factor space, even though they do not form two dis- 
tinct clusters. In this case the vectors present à fan-shaped configuration 
by means of which the reference plane is defined. The two reference axes 
in this plane can be selected with much greater freedom than when the axes 
are restricted to the clusters of vectors. 

The equivalence of the geometric and algebraic interpretation of a statis- 
tical variable will now be pointed out. As shown in Chapter II, a variable 
may be considered as a linear function of, say, m common factors. Geo- 
metrically, this means that the vector representing the variable lies in an 
m-space, which is defined by the m reference axes representing the factors. 
Thus if а vector lies in a plane, it is describable in terms of two reference 
vectors, and hence in the algebraic description of the variable only two 
common factors appear. Of course, other variables of the set might in- 
volve these or other factors. 

c) Hyperplanar fit.—In the two preceding standards one-spaces and two- 
spaces were the bases for determining the adequacy of geometric fit. This 
idea can be extended to higher spaces. By hyperplanar fit in a space of m 
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dimensions is meant that each vector representing a variable in the com- 
mon-factor space lies in an (m — 1), or smaller, space. 

When a set of variables satisfies this standard, the complexity of any one 
of them is less than the total number of common factors. This does not ap- 
pear as a very stringent criterion at first sight, because it is satisfied if each 
variable is of complexity (m — 1) for m common factors. The strength of 
this standard, however, lies in the fact that the hyperplane is the largest 
permissible space containing each variable. In other words, it is hoped that 
there will always be smaller reference spaces which contain certain sub- 
groups of variables. Thus the complexities of the variables are reduced be- 
low (m — 1). In particular, a vector which lies on a reference axis is con- 
tained in that one-space, and the variable it represents has a complexity of 
one. Similarly, if a vector lies in a reference plane, the variable is of com- 
plexity two. This analogy of the smallest reference space in which a vector is 
contained and the complexity of the variable can be extended to any degree. 

It is evident that the two preceding geometric standards may be con- 
sidered as special cases of hyperplanar fit. For, if a set of variables satisfies 
the criterion of linear fit, or planar fit, it certainly conforms to hyperplanar 
fit. The converse, of course, is not true generally. Hence the ideal to be 
aimed for is to reduce the hyperplanar fit to a geometric fit of as small a 
number of dimensions as possible, the limit being linear fit. 


6. GEOMETRIC FIT: POINT REPRESENTATION 

Іп 3.7 two alternative geometrie representations of a set of variables 
were presented. By employing the vector representation, three standards 
were immediately evident which will lead to as many types of preferred 
factor patterns. Now, by considering the point representation, another 
standard evolves. It will be recalled that in this representation there is one 
point for each of the N individuals, referred to asystem of n reference axes— 
one for each variable. The points which are plotted in this n-space are con- 
tained in a common-factor space of only m dimensions.* The loci of the 
swarm of points of uniform frequency density are, more or less, concentric, 
similar, and similarly situated m-dimensional ellipsoids, being exactly so 
for a normally distributed population.+ It then seems natural to take the 

ж According to Theorem 3.2, the N points, (zu, 22, . «+, Zm), (212, 222) -+ >; Za) 55 
(zs, Zax, 222) ах), Whose coordinates are the elements in the columns of the matrix 
|211, are all contained in a linear m-space, where m is the rank of this matrix. Then the 


argument of 3.8 indicates that the common-factor space (containing the N points) is of 
m dimensions, where m is the rank of the correlation matrix, with communalities in the 


diagonal. 
1 б. Udny Yule and M. С. Kendall, An Introduction to the Theory of Statistics, 1937, 


Chap. XII. 
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It has been shown in 3.7 and 3.8 that the variables may be regarded as 
vectors with а common origin and that the correlations between variables 
are given by the cosines of the angles between such vectors in the N-space, 
or by the scalar products of the projected vectors in the common-factor 
space. А group of variables yielding a cluster of high intercorrelations is 
thus encompassed by a “cone” with a relatively small generating angle. If 
a reference axis or vector of the common-factor space is chosen in the midst 
of this cone, all variables in the group will correlate high with it. To a fac- 
tor common to a group of variables, there thus corresponds a reference vec- 
tor. The degree of linear fit is measured by the compactness of the “cone” 
representing the particular group of variables, or the extent to which the 
vectors of these variables approach the axis of reference. By selecting а 
number of such reference axes, each one passing through a cone of vectors, 
the whole configuration may finally be well fitted. 

Тһе standard of linear fit, together with that of uncorrelated factors, 
usually can be met only roughly in the case of positive correlations among 
the variables. It is evident that a closer linear fit can be obtained by per- 
mitting the factors to be correlated, as will be shown in Chapter XI. 

b) Planar fit.—The type of geometric fit just described may also be in- 
terpreted as a planar fit. Each plane is defined by two of the reference axes, 
or by the end points of the two reference vectors and the origin. Good geo- 
metric fit is then indicated by the proximity of the vectors representing the 
variables in the common-factor space to such planes. 

A subset of variables may be well represented by vectors in a reference 
plane of the common-factor space, even though they do not form two dis- 
tinct clusters. In this case the vectors present a fan-shaped configuration 
by means of which the reference plane is defined. The two reference axes 
in this plane can be selected with much greater freedom than when the axes 
are restricted to the clusters of vectors, 

| Тһе equivalence of the geometric and algebraic interpretation of a statis- 
tical variable will now be pointed out. As shown in Chapter II, a variable 
may be considered as a linear function of, say, m common factors. Geo- 
metrically, this means that the vector representing the variable lies in an 
m-space, which is defined by the m reference axes representing the factors. 
Thus if a vector lies in a plane, it is describable in terms of two reference 
vectors, and hence in the algebraic description of the variable only two 


common factors appear. Of course, other variables of the set might in- 
volve these or other factors. 


c) Hyperplanar fit.—In the two preceding standards one 
spaces were the bases for determining the adequacy of ge 
idea can be extended to higher spaces. By hyperplanar fi 


-spaces and two- 
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dimensions is meant that each vector representing a variable in the com- 
mon-factor space lies in an (m — 1), or smaller, space. 

When a set of variables satisfies this standard, the complexity of any one 
of them is less than the total number of common factors. This does not ap- 
pear as a very stringent criterion at first sight, because it is satisfied if each 
variable is of complexity (m — 1) for m common factors. The strength of 
this standard, however, lies in the faet that the hyperplane is the largest 
permissible space containing each variable. In other words, it is hoped that 
there will always be smaller reference spaces which contain certain sub- 
groups of variables. Thus the complexities of the variables are reduced be- 
low (m — 1). In particular, a vector which lies on a reference axis is con- 
tained in that one-space, and the variable it represents has à complexity of 
one. Similarly, if a vector lies in a reference plane, the variable is of com- 
plexity two. This analogy of the smallest reference space in which a vector is 
contained and the complexity of the variable сап be extended to any degree. 

It is evident that the two preceding geometric standards may be con- 
sidered as special cases of hyperplanar fit. For, if a set of variables satisfies 
the criterion of linear fit, or planar fit, it certainly conforms to hyperplanar 
fit. The converse, of course, is not true generally. Hence the ideal to be 
aimed for is to reduce the hyperplanar fit to a geometric fit of as small a 
number of dimensions as possible, the limit being linear fit. 


6. GEOMETRIC FIT: POINT REPRESENTATION 

In 3.7 two alternative geometric representations of a set of variables 
were presented. By employing the vector representation, three standards 
were immediately evident which will lead to as many types of preferred 
factor patterns. Now, by considering the point representation, another 
standard evolves. It will be recalled that in this representation there is one 
point for each of the N individuals, referred to a system of n reference axes—- 
one for each variable. The points which are plotted in this n-space аге con- 
tained in a common-factor space of only m dimensions.* The loci of the 
Swarm of points of uniform frequency density are, more or less, сопсепітіс, 
Similar, and similarly situated m-dimensional ellipsoids, being exactly so 
for a normally distributed population.f It then seems natural to take the 


* According to Theorem 3.2, the N points, (zu, 221) + + +» Zu), (212, 29, . © y Zna) e+ 
(ањ, zon, . , Zay), whose coordinates are the elements in the columns of the matrix 


llessll, are all contained in a linear m-space, where m is the rank of this matrix. Then the 
argument of 3.8 indicates that the common-factor space (containing the N points) is of 
m dimensions, where m is the rank of the correlation matrix, with communalities іп the 
diagonal. 


1 G. Udny Yule and M. G. Kendall 
Chap. XII. 


, An Introduction to the Theory of Statistics, 1937, 
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prineipal axes of these ellipsoids as the fundamental reference axes. This 
standard, which is called ellipsoidal fit, leads to another preferred type of 
factor pattern.* 

From a statistical point of view several of the preferred patterns may fit 
a matrix of observed correlations equally well. The ultimate choice of type 
of factor pattern must then rest, in part, upon the nature of the variables 
and the utility of the solution in the particular field of investigation. 


5.3. Algebraic Solution of Any Symmetric Matrix 


Before describing the various preferred types of factor patterns, a solu- 
tion of special mathematical and historical interest will first be exhibited. 
This solution is obtainable by means of a general algebraic procedure for 
factoring any symmetric matrix, known as “completing the square.” The 
method was applied specifically to a correlation matrix by MeMahonf be- 
fore 1923. In the early stages of factor analysis one writer] developed the 
solid staircase method, while another§ also treated this type of solution 
under the title diagonal method. 

Тһе n variables may be described in terms of n (or possibly fewer) new 
uncorrelated factors, by means of the above methods, in the following form: 


2 = au, 
Zo = dal’) + а» з, 
(5.1) 23 = аз. + ав + as, 


Zn = аа. + Anos + аз: +... T Ganini 


It is evident that a great many variations of this particular form of solution 
are possible, since any one of the variables may be selected as involving only 
one factor. A solution of the form (5.1) is therefore undesirable, since it is 
not definitive enough. 

Тһе preferred types of solution that аге presented in the remainder of 
this chapter are well defined inasmuch as they conform closely to certain 
of the standards discussed in 5.2. By adhering to the assumptions underly- 


ing any one of the preferred types of pattern, a single solution may be ob- 
tained from a given set of correlations. 


* The application of higher-dimensional ellipsoids in the field of factor analysis was 
proposed by Truman L. Kelley at a meeting of the Unitary Traits Committee in 1933. 


1 James MeMahon, “Нурегврһегіса! Goniometry; and Ив Application to Correlation 
Theory for n Variables," Biometrika, 1923. 


а } K. J. Holzinger, Preliminary Report on Spearman-Holzinger Unitary Trait Study, 
0. 5. 


§ L. L. Thurstone, The Vectors of Mind, 1935, pp. 78-81. 
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5.4. Uni-factor Pattern 

The first of the preferred patterns to be discussed may be regarded as an 
ideal, since it is highly improbable that it will be attained with actual data. 
This, as well as all the other preferred types, is in agreement with the as- 
sumed composition of statistical variables. Another property which will 
hold true for all solutions in this chapter is that the factors are uncorrelated. 
The additional assumptions, which distinguish the particular pattern under 
discussion, are those of best geometric fit and minimum complexity. In the 
vector representation of variables the best geometric fit is attained when the 
projected vectors (into the common-factor space) of a group of variables lie 
close to a reference axis. When the vectors lie exactly on such axes, each 
variable then measures but a single factor. Of course, variables of distinct 
groups measure different factors. 

Such an ideal solution, which satisfies the standard of linear fit and hence 
minimum complexity, will be called a uni-factor pattern. This type of pat- 
tern will now be discussed in detail. Let there be n variables and m group 
factors (m < n) such that p: variables are expressible as linear functions-of 
only the first factor Ға, рг other variables are expressible as linear functions 
of the second factor Fs, and so forth, to the remaining pm variables which 
are expressible as linear functions of only the last factor Fm. Then variables 
may be subdivided into m groups Gi, Gs, . . . , Gm according to the factor 
involved in each variable, so that n = (pi + pa +... + Pm). Then, if the 
variables are numbered consecutively from 1 to n, such a uni-factor pattern 
(omitting unique factors) may be written as follows:* 


д = an, 


25 = an^, 


= ap, 


(5.2) ғыт арына; 


Фар = афұр,2 Ға, 


а-ы = O(n Dt inl m , 
fac Ола n 


* А simplification in notation is employed in this and all succeeding factor patterns. 
Since the linear expressions аге approximations to observed variables, they should be 
denoted by 2; in accordance with (2.4). Also when unique factors are omitted, the vari- 
ables may be considered as projected into the common-factor space and thus be repre- 
sented by z/^. For simplicity, however, all primes are dropped, and it is understood that 
any linear expression stands for an approximation to the indicated variable. 
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It is assumed in this pattern, and the two succeeding ones, that the coeffi- 
cients of the factors are positive. The contributions of the factors are evi- 
dently dependent on the number of variables in the subgroups which meas- 
ure the respective factors. When these subgroups are fairly equal, the con- 
tributions tend to be uniform. The uni-factor pattern may therefore be said 
to satisfy standard 4b of section 5.2. 

А simple schematic representation of a uni-factor pattern is given in 
Table 5.1. The vertical lines represent the coefficients of the respective fac- 
tors. Such a diagram then indicates clearly the mutually exclusive charac- 
ter of the subgroups of variables. 

TABLE 5.1 
UNI-FACTOR PATTERN SCHEME 


Variable Р\ Ез Ға 25 Ғы 


To distinguish between variables of different groups, the standard set- 


theory notations* can be adapted to the present needs. Тһе primary defini- 
tions follow: 


а) eeG, means e is a variable in the group С,. 

B) (eg Jey, $2 1,2, ..., т) denotes the system of elements 2; 
for all values of j in groups G,, where the range of s is in- 
dicated. Тһе elements of the system are first designated, 
followed by a semicolon, and all the properties on the elements 

(5.3) are to the right of the semicolon. 

c) (ан; = 1, 2,..., М) means the sum of all the elements іп 
the system (2; = 1,2,..., М). The index j is fixed, the 
summation extending over, 1. This sum is equivalent to the 


more conventional form > 235. 


ісі 


* See, for example, L. R. Wilcox, “Modularity in the Th ғ ТИИТ" 
Mathematics, 1939. н y eory of Lattices," Annals of 
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It will be found that these definitions* aid in clarity and ultimate simplicity 
in deseribing much of the following theory. 


Since the factors are always assumed to be in standard form, the correla- 
tion coefficient between any two variables jeG, and Кеб, is reproduced by 
means of pattern (5.2) as follows: 


1 қ " 
тк = ў (2:2; 4 = 1,2,..., №) 


(5.4) 


1 


Ты. В 
N DapF ul; i= 1,2,..., № 
= аак, · 


When the two variables belong to different groups, the reproduced correla- 
tion becomes 


(5.5) rj —0 (jeG, , keG,, sz 0, 


since the factors are uncorrelated; whereas, if the variables are in the same 
group, 


(5.6) Tik = анайы (j,keGs) . 


From formulas (5.5) and (5.6), it is seen that a factor pattern of type (5.2) 
can only reproduce a set of correlations of the form shown in Figure 5.1. 
The plus signs in the small triangles represent positive correlations among 
variables within groups, and the zeros in the rectangles represent lack of 
correlation between variables in different groups. It is unlikely that such a 
picture of intercorrelations of variables would be obtained in practice, and 
hence a pattern of type (5.2) has been termed “‘ideal.”” To the extent to 
which a matrix of observed correlations approaches the above form, its solu- 
tion may be considered as approaching the uni-factor pattern. 

The only practical instance in which a solution conforms to the uni-factor 


* To illustrate the definitions given in (5.3), consider a set of ten variables (zi, 22,..., 
710), grouped as follows: 
Gi = (1,2,3, 4,5), @ = (6,7,8), Gs = (9, 10). 
Ап instance of (а) is 7€G;, which means that variable 7 is included in group G;. The sys- 
tem of elements (z;; jes, s = 1, 3) is the set of variables (zi, 22, 23, 24, 25, Zs, 210). The sum 
of N values of variable 4, which may be taken as ап example of (c), is 
х(аш;і4-1,2,..., М). 


n explicitly for the individual values of the variables, 


T Pattern (5.2) could be writte 
that is the form implied in the development of (5.4). 


ав was done in eq. (2.5). Actually, 
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form is the classical “Two-factor” pattern.* This solution is called “Two- 
factor” by Spearman because two types of factors—general and unique— 
are involved. From the point of view of complexity, however, this solution 
is a uni-factor one, since each variable involves just one common factor (the 
same one). To illustrate a “Tywo-factor” pattern, suppose that four vari- 


о 
о 


о 
о 


Fia. 5.1 


ables satisfy the tetrad conditions (4.6) for the existence of just one common 
factor. Then the pattern may be written in the form: 


а = ayFo + aUi, 


(5.7) Z2 = aF о + aU2, 
Z3 = Asolo + a3U3, 
24 = asFo + aUa. 


It will be noted that this pattern is a special case of (5.2), namely, for only 
one factor. The correlations of a set of variables leading to a “Two-factor” 
pattern correspond to those in one triangle of Figure 5.1, which satisfy the 
general conditions (4.13) for one general factor. 


5.5. Bi-factor Pattern 


The next type of factor solution to be discussed is a modification of the 
preceding one. As already noted, a uni-factor pattern can only be produced 
from a matrix of correlations of the form indicated in Figure 5.1, and which 
is highly improbable of attainment. In a practical case, when a set of rele- 


* Charles Spearman, The Abilities of Man, 1927. 
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vant variables is employed to study the common attributes of a group of 
individuals, it is to be expected that the correlations between subgroups of 
such variables will be positive throughout rather than zero. Still higher 
correlations will usually be found for the variables within subgroups of the 
entire set. Such a matrix of correlations is indicated schematically in Fig- 
ure 5.2. The need for modifying the uni-factor type of pattern when dealing 
with observed data is thus apparent. 


€—M 5 


Fia. 5.2 


If, in addition to the distinct group factors postulated in the uni-factor 
pattern, a general factor is also included, the positive correlations through- 
out the observed matrix can be accounted for. This modification of the uni- 
factor pattern is tantamount to substituting planar for linear geometric fit. 
When the vectors of a group of variables lie in a reference plane, each vari- 
able measures just two factors. If, furthermore, the vectors lie only in the 
reference planes formed by à general-factor (Fo) axis and one group-factor 
axis (none in the planes of two group factors), the configuration can be de- 
scribed as a pencil of planes through the Fo axis.* A solution satisfying 
standards 1, 2, 3, 4c, and planar fit, as here described, will be called a bi- 
factor pattern. 

* In ordinary space geometry & pencil of planes refers to the totality of planes through 
a line, i.e., all the planes linearly dependent on two distinct planes through the line. In 
the present setting, however, & pencil of planes in a space of (m + 1) dimensions will 
refer to all the planes through а line which are mutually orthogonal. It is clearly seen 
that there are m such planes in the pencil through the Fo axis. А 

T The bi-factor pattern actually becomes & uni-factor pattern when the factors are 
correlated, as will be shown in 11.3. 
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Тһе formal presentation of the bi-factor pattern will now be given. Let 
the n variables be expressible as linear functions of (m + 1) common factors 
as follows: every variable involves a general factor Fo, and, in addition, pi 
variables involve the first group factor /i, рә other variables involve the 
second group factor Ез, and so on to the remaining p, variables which in- 
volve the last group factor Fm. The n variables may be rearranged accord- 
ing to the groups Gi, Gs, . . . , Gm, and numbered consecutively from 1 to n. 
Then, employing the notation of the preceding section, the bi-factor pattern, 
omitting unique factors, may be written in the form 


2 = aio + aul, , 
2 = азо + ал) Й 
Zp, = Apolo + ag, 
2541 = Ap, +1)0F0 + аф +2, 
(5.8) 
Zpyep, = p +p,)0!"o + ашар, 
Zn—-pm+1 = (sp, i) o + (раа i 
= go T 
Zn = алһоЁо Чй 


A schematic plan of the bi-factor pattern, similar to that given for the 
uni-factor pattern, is presented in Table 5.2. It is apparent from this dia- 


TABLE 5.2 


Variable Fo Fy Рз BE $e Ғы 
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gram that each of the group factors overlaps with the general factor but 


not with any of the others. 
The correlation between any two variables jeG, апа keG, may be repro- 


duced by means of a bi-factor pattern as follows: 


j 1 a 
Тук = ру 20292) і-1,2,..., М), 


(5.9) збан + аР, Шашы + амға; # = 1,9,..., М), 


= аак 4 аранҒғ,ғ, + Gisele e, + GRIFF, з 
= аук + Gt F,F, 5 


where the last equality follows from the fact that the general factor is un- 
correlated with the group factors. If the two variables are in different 


groups, then 


(5.10) тк = (0;000 (jeG, , keG, › $87 t) , 


but, if they are in the same group, their reproduced correlation becomes 


(5.11) Tj, = айко F 40 (7,Кеб,). 

From equation (5.10) it is evident that the relatively small correlations 
between variables of different groups are produced through the general fac- 
tor Fo. These correlations appear in the rectangles of a correlation matrix 
as in Figure 5.2. The higher correlations between variables within a group 
аге reproduced by formula (5.11), which involves the general and one group 
factor. Such correlations are represented in the triangles of Figure 5.2. 

The form of the bi-factor pattern was conceived* as a direct extension of 
Spearman's "T wo-factor" solution in the case of sets of variables which do 
not meet the assumptions underlying the latter solution. For a specialized 
set of variables the ‘“Two-factor” pattern, i.e., one general factor, may be 
sufficient to reproduce the correlations. With a complex set of variables, 
however, the criteria for a single general factor may not be satisfied, and 
group factors may be required. The bi-factor solution makes allowance for 
both general and group factors. 


5.6. Multiple-Factor Pattern 
An alternative solution to the bi-factor pattern which accounts for a table 
of essentially positive correlations, but which does not involve a general 


Spearman-Holzinger Unitary Trait Study, 


*K. J. Holzinger, Preliminary Report on 
т d by Frances Swineford 


No. 5. A pattern essentially of the bi-factor form was propose 
during the preparation of Report 5. 
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factor, will be called а multiple-factor pattern.* In this solution the next 

stage of geometric fit, viz., hyperplanar, is employed. Analytically, this 

means that a number of “overlapping” group factors must be postulated in 
order to account for the generally positive correlations among a set of vari- 
ables, as exhibited in Figure 5.2. These factors are called "overlapping" to 
distinguish them from the distinct group factors which are employed in the 
pattern (5.2). The term "overlapping" indicates that several factors appear 
in the description of a variable. Thus, in the bi-factor pattern the group 
factors overlap with the general factor, whereas in the multiple-factor pat- 
tern the group factors overlap among themselves. Overlapping thus implies 
a complexity of two or more. By sacrificing the general factor of a bi-factor 
pattern, a higher complexity than two will usually be required in a multiple- 
factor pattern. It is for this reason that the prefix “multiple” is employed. 

A multiple-factor pattern satisfies standards 1, 2, 3, 4b, and 5c. The 
standards of parsimony and hyperplanar fit may be stated in analytical 
terms. In conformity with these standards, it is desirable that the com- 
plexities of the variables in a multiple-factor pattern be as small as possible; 
and, in spite of the necessary overlapping of factors, it is also desirable that 
there be as many zero coefficients in the columns as possible. If these con- 
ditions are satisfied, then the factors can be identified by means of the 
groups of variables with appreciable coefficients, 

For a given table of correlations there sho 
pattern with a large number of zero coeffic 
configuration of zeros and overlap of group f; 
ferent sets of variables. Hence no general m. 
multiple-factor pattern is possible apart fro 
Therefore, no schematic plan for this type 


uld be a unique multiple-factor 
ients. However, the particular 
actors will usually vary for dif- 
athematical formulation of the 
m a given matrix of correlations. 
of solution is presented. An illus- 

* Although a multiple-factor pattern is here 
in actual practice it must be derived by rotati 


such preliminary pattern, known as the centri 
Chap. VIII. 


formulated as one о 


f the preferred types, 
on from some 


а preliminary solution. One 
oid pattern, will be described in detail in 


t Such а configuration, with 
simple structure by Thurstone ( 
satisfied: 


correlated or uncorrela 


e ted factors, has been called a 
op. сй., Chap. VI) whe 


n the following conditions are 

1. Each row of the factor structure V (see 2.4) should have at least one zero 

2. Each column of V should have at least m zeros (m being the total number of com- 
mon factors) 

3. For every pair of columns of 


vanish in one column but not in the other 


With special reference to psychology, Thurstone has called the factors determined by 
hyperplanar fit “primary abilities,” 
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tration of a possible form of a multiple-factor pattern, however, is given for 
a special case of twelve variables and four factors: 


а = а + aF: + auF, 


2» = aaFi + аР; , 

23 =anF, + аР + аз, 

24 = аһ; " Tau, 

25 = аР + аз Ез + aaki, 
(5.12) % = аР; + аР, А 

по аР. + аз + auka, 

28 = а + ањ , 

2 = аР + asks + anf, А 

Zio = aii, + аша”, 

Zu = ац, Wy + an, aF; " 

Z2 = ар, + ањ, з + ало, 4, . 


Аз іп the case of the uni-factor pattern, the contributions of thése factors 
are approximately level. From such a pattern, it is obvious that the correla- 
tions are reproduced by the general formula (2.22). Every pair of variables 
is linked by overlapping group factors and thus yields a positive correlation. 
А given matrix of correlations, of the form in Figure 5.2, can thus be re- 
produced by a multiple-factor pattern. 

Although the three preferred types of patterns are formally distinct, with 
actual data the resulting solution may be a combination of them to a cer- 
tain extent. Thus a solution may be of the bi-factor form but have some 
overlap of group factors and, to this extent, resemble a multiple-factor solu- 

tion. Likewise a pattern of the multiple-factor type may have a factor ap- 
pearing essentially in all variables and thus resemble the bi-factor form. 
Either the bi-factor or the multiple-factor types may involve several vari- 


ables which approximate unit complexity and thus partially approach the 
ideal uni-factor form. 


5.7. Principal-Factor Pattern 


А The preferred types of factor patterns exhibited thus far are of a special 
kind, that is, the; 


y require that all factor coefficients be positive, and the 
multiple-factor pattern furthermore precludes а general factor. If a matrix 
of correlations is Observed to be in the form of Figure 5.1, there is no doubt 
but that a uni-factor solution is most meaningful. On the other hand, when 
a table of correlations appears as in Figure 5.2, there is no assurance, from 
the statistics alone, that a bi-factor or multiple-factor solution will afford 
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the most useful interpretation of the variables. The nature of the data may 
then aid in the selection of a particular form of representation. It is con- 
ceivable that certain sets of variables may be better interpreted in terms of 
factors, the coefficients of which are not restricted to be positive. 

A factor for which several of the coefficients are positive and others are 
negative will be called a bipolar factor.* Such a factor may appear in all 
variables of a set or only in a subgroup of them. А bipolar factor is not es- 
sentially different from any other but is merely one for which several of the 
variables have significant negative projections. Such variables may be re- 
garded as measuring the negative aspect of the usual type of factor. Thus, 
if a number of variables identified with “fear” are represented by positive 
projections, variables with negative projections might be interpreted as 
measuring “courage.” It would appear simpler, however, to regard the fac- 
tor merely as "fear," and the opposing set of variables as measures of “пера- 
tive fear." Of course, the signs of all the coefficients of the factor may be 
changed without altering the adequacy of the solution. Such reversal in the 
foregoing example would lead to the interpretation of the factor as "'cour- 
age," and the subgroup of variables with negative coefficients would be re- 
garded as measuring "negative courage." In the illustrations of the text a 
single name for a bipolar factor will be employed. This is consistent with 
representing any factor by a single continuum. 

Тһе preferred type of pattern now to be discussed involves bipolar factors 
which are present in all variables of the set. In addition to these bipolar 
factors, there is also а general factor with positive coefficients throughout. 
Such a solution which satisfies standards 1, 2a, 3, 4a, and 6 is called a prin- 
cipal-factor pattern. 

The criterion of ellipsoidal fit determines the essential nature of the refer- 
ence system. For then the factors are represented geometrically by the 
principal axes of the ellipsoids described in standard 6. The corresponding 
analytie standard is given by 4a, since the factors contribute decreasing 
amounts to the total communality. Thus the first factor is one with refer- 
ence to which the individuals of the sample show the greatest possible 
variability. The remaining factors contribute successively smaller vari- 
ances. 


* This term was introduced by Cyril Burt in “Тһе Factorial Analysis of Emotional 
Traits,” Character and Personality. In the present text, however, the term is used in а 
more general sense. 


1 Harold Hotelling devised a solution of this form but factored the total unit variance 
or the reliability of each variable instead of the (estimate of) communality (cf. “Analysis 
of а Complex of Statistical Variables into Principal Components," Journal of Educational 
Psychology, 1983). 
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А principal-factor pattern, in which the unique factors have again been 
omitted, may be exhibited as follows: 


2р = а, + diss + аһЁз Bi. аы.» 

Z2 = daly + аЕ + assa +... + аза т, 
(5.13) Z3 = agF + а» + аззЁз +... аза ы, 

Zn = Only + аР + ааа Ps es g TTAN 
In such a pattern the complexity of each variable is equal to the total num- 
ber of common factors. The first factor is an ordinary general factor whose 
coefficients ал (j = 1, 2, . . . , n) are all positive when the solution is based 
upon a table of positive correlations. On the other hand, approximately 
half the coefficients of each of the remaining factors are negative, that is, 
Fo, F3,..., Fm are bipolar factors. 

Obviously a pattern of type (5.13) can reproduce a general table of corre- 
lations like those exhibited in Figure 5.2. The solution is thus perfectly 
satisfactory from a statistical point of view. Although a principal-factor 
pattern may be more complex than the other preferred types, it may some- 
times furnish a more convenient representation of a particular set of vari- 
ables. 

Since the uni-factor, bi-factor, and multiple-factor patterns involve only 
positive coefficients, they can reproduce a table of positive correlations only. 
In actual analysis, therefore, these types are adapted only to sets of vari- 
ables which are positively correlated or to sets which can be put in that 
form. If there are a few insignificant negative correlations, the preceding 
types of solutions can be obtained. If a variable has significant negative 
correlations with all the rest, its scale may be reversed, thereby changing 
the signs of the correlations. One such example is that in which a measure 
of stupidity is changed to that of intelligence by changing the sign of the 
Scale. By this procedure of “reflection of variables" a matrix of essentially 
positive correlations might be obtained. 

If, after such sign-changing, significant negative and positive correlations 
appear for some of the variables, the first three preferred types would not 
be applicable. Since the principal-factor pattern is not restricted to positive 
Coefficients, it can reproduce negative as well as positive correlations. This 
type of solution may then be applied to any matrix of correlations. An ex- 
ample of a principal-factor solution based upon positive and negative corre- 
lations is given in 7.5. In this solution all factors, including the first, are 
of the bipolar form. 
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5.8. Summary of Preferred Patterns 


It will be noted that the names "uni-factor," *tbi-factor," and **multiple- 
factor" indicate the complexities of the variables in the respective patterns. 
Thus in the uni-factor pattern each variable is of complexity one, while in 
the bi-factor pattern each variable is of complexity two. In the multiple- 
factor pattern the complexities of the variables may differ greatly, but no 
variable involves all the factors. As already indicated, the principal-factor 
pattern takes its name from the property of the reference system rather 
than from the complexities of the variables, which are generally equal to 
m for m common factors. In recapitulation the various assumptions 
and properties of the four preferred types of factor patterns are given in 
Table 5.3. 


TABLE 5.3 
ASSUMPTIONS AND PROPERTIES OF PREFERRED PATTERNS 
ma 1 
Type of Assumptions MR vea Distinguishing 
Pattern (Enumerated in 5.2) esters Variable Characteristics 
Тіпі-Ғасбог........... 1, 2а, 2b, 3, 4b, 5a m 41 Distinct group 
А faetors 
Bi-faetor.. sisi se = sn ora 1, 2a, 2b, 3, 4c, 5b | mor m+1 2 One general plus 
М group factors 
Multiple-factor....... 1, 2a, 2b, 3, 4b, 5c m «m Overlapping 
Жк group factors 
Principal-faetor....... 1, 2a, 3, 4a, 6 m m One general plus 
bipolar factors 


From the above discussion it is apparent that the uni-faetor pattern is 
the ideal sought in all factor analysis. The remaining preferred types meet 
various standards for acceptable solutions, but these criteria do not furnish 
the complete basis for a choice of form. This choice will also depend upon 
the nature of the variables and the theories or laws in a particular field of 
application. Thus, if an investigator considers the variables to be of the 
“bipolar” type, the principle-factor form would be appropriate. A simple 
example of such a set of variables, in the field of biology, is given by stand- 
ing height, sitting height, and length of forearm as opposed to weight, chest 
girth, and chest width. Ifa general factor were denied by a theory in a par- 
ticular field, then the multiple-factor type would be consistent; but, if the 
general factor were accepted in such a theory, then the bi-factor form would 
be suitable. Such contrasting theories now exist in the field of psychological 
abilities, but a theory including a general factor is commonly accepted for 
physical traits in biology. According to another theory, an analysis might 
be desired in which the factors appear in the order of their importance. This 
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type of importance is indicated by the decreasing contributions of successive 
factors of the principal-factor solution. It is thus evident that general scien- 
tifie, as well as statistical, standards in any field of investigation must be 
employed in the selection of а preferred pattern. 

It will be shown in Part III that, once estimates of communalities have 
been made for a given solution, the latter may be converted into any other 
form desired by a suitable transformation, and all such solutions will fit the 
correlations equally well. Preference for a given form must thus depend 
upon statistical standards, such as those discussed in this chapter as well as 
criteria apart from the statistics. Examples of such criteria are Spearman's 
psychological theory of a general factor generalized to the bi-factor form 
and Thurstone's contrasting theory of primary abilities in which the general 
factor is not included. 

In psychology, or any other field, an investigator is free to postulate a 
theory and show that it is compatible with experimental evidence. It is 
possible, however, that another investigator may postulate an alternative 
theory also compatible with such evidence. The mathematical expressions 
(the patterns) of these theories may be formally different because the stand- 
ards employed in the two cases would not be the same. Inasmuch as both 
theories are consistent with empirical evidence, neither should be used аза 
basis for judging the appropriateness, or inappropriateness, of the other. 
This type of conditioned thinking has been used in factor analysis, however, 
leading to much confusion as to the ultimate form of solution to be adopted. 

In the application of factor analysis to psychology it has been argued 
that one form of solution is incorrect because it is inconsistent with another 
form. In particular, the principal-factor form of solution has been rejected 
on the grounds that it depends upon the nature of the test battery, while 
another form of solution has been accepted because it is argued that the 
latter is invariant for the battery. It should be apparent, however, that all 
solutions are dependent on the particular set of variables analyzed, and any 
argument for choice of solution cannot logically depend upon such assumed 
invariance. 

The form of any factor solution, ultimately selected, depends upon the 
following features: (a) the group of individuals measured; (b) the set of 
variables and all their intercorrelations; (c) statistical standards, such as 
those in the present chapter; and (d) outside criteria from the particular 
field of investigation. Like all other statistics, factor analysis is obviously 
dependent upon the particular sample of individuals, and generalizations 
шау be made by conventional methods. It is also apparent that factor 
analysis is a correlational analysis and, therefore, is dependent upon all the 
variables producing the correlations. If additional variables are added to a 


E 


108 FACTOR ANALYSIS 


given set, the table of correlations, as a whole, will usually change. Inas- 
much as any factor solution is based upon all the intercorrelations, it will 
vary as the set of variables is changed. Practically, of course, the addition 
of only a few variables may not change the resulting solution appreciably. 

Тһе dependence of a factor solution upon the groups and the variables is 
analogous to the dependence of a multiple regression equation upon the 
sample and the particular variables used for estimation. Statistical stand- 
ards and criteria from a given field are introduced so as to obtain a determi- 
nate solution in harmony with a postulated theory. 

The above considerations should be clearly evident if factor analysis is 
viewed as a statistical method yielding solutions which are convertible from 
one to another. Because of their statistical nature, factor solutions will, in 
general, not be invariant, and because of their convertibility a preference of 
form сап be made only upon employing appropriate outside criteria. In fac- 
tor analysis, as in all empirical sciences, several equally satisfactory laws 
may be usefully employed, although they may be formally quite different.* 

In the field of psychology attempts have been made to formulate invari- 
ant solutions. The theory underlying such invariant factors involves the 
arbitrary specification of the four aspects upon which factor analysis de- 
pends. Thus by fixing the population and the set of variables, and agreeing 
upon the form of solution, a fundamental set of factors may be obtained. 
Then all other variables in the given field may be expressed in terms of these 
factors. 

After the methods of solution for various types of factor patterns have 
been presented, the above points will be discussed in connection with the 
choice of form of solution for practical problems (see Chap. XIII). The prob- 
lem of invariance, or stability of solutions, will again be considered and il- 
lustrated with actual data. 


* A somewhat analogous situation arises in that branch of physics known as hydro- 
dynamics. A clear discussion of this problem is given by Horace Lamb as follows: “Тһе 
equations of motion of a fluid have been obtained in two different forms, corresponding to 
the two ways in which the problem of determining the motion of a fluid mass, acted on by 
given forces and subject to given conditions, may be viewed. We may either regard as 
the object of our investigations а knowledge of the velocity, the pressure, and the density, 
at all points of space occupied by the fluid, for all instants; or we may seek to determine 
the history of every particle" (Hydrodynamics, p. 2). 

The equations obtained according to these two approaches are called the “Eulerian” 
and the “Lagrangian” forms of the hydrokinetic equations. In like manner, it has been 
the custom in factor analysis to label a particular form of factor solution after the man 
who has done most toward its development. 


PART II 
DIRECT ORTHOGONAL SOLUTIONS 


CHAPTER VI 
THE BI-FACTOR SOLUTION AND SAMPLING FORMULAS 


6.1. Introduction 


The foundations and formal groundwork for factor analysis have been 
laid in the preceding chapters. Now the analytical methods that may be 
employed in the solution of a practical problem will be developed in detail. 
In the present chapter the method of obtaining a bi-factor pattern for a set 
of variables will be discussed. 

To pave the way for a bi-factor analysis, the complete solution of Spear- 
man's "Two-factor" pattern is given; first, in the classical manner in 6.2, 
апа then by a revised method. The assumptions underlying either method 
of solution are examined in detail so that the validity of a '"Two-factor" 
pattern сап be clearly judged. In 6.4 the solution for a bi-factor pattern is 
developed. The plan of Chapter V is adopted, and the methods of 6.2 and 
6.3 are adjusted, to this more complex situation. When the simple plan (5.8) 
cannot be met by a particular set of variables, certain modifications must 
be made. Such revisions and verification of the simple bi-factor pattern are 
considered in 6.9. 

To aid in the judgment of the adequacy of any factor solution, certain 
sampling error formulas are necessary. The first of these—for the standard 
error of a general-factor coefficient—is developed in 6.5. Then a formula for 
the standard error of a residual is worked out for the case of any number of 
factors. This formula, however, involves the standard errors of the pattern 
coefficients. Additional sampling error formulas, based upon further ap- 
proximations, are presented in 6.7. Then, to facilitate the computation in 
ап actual problem, tables of their values are given in Appendix H. 

To illustrate the analytical methods of this chapter several numerical 
examples are employed. А set of five physical variables is used to clarify 
some of the ideas in connection with the sampling error formulas. The bat- 
tery of twenty-four psychological tests, which has been introduced іп Chap- 
ter II, is employed in 6.10, where the bi-factor solution is exhibited. The 
complete outline for the computation of this pattern is given in Appendix C. 
It is believed that the procedure for calculation will be useful for reference 
purposes after the reader has become acquainted with the logical basis of 
the method, and it was therefore separated from the text and put in the 
appendix. 

111 


112 FACTOR ANALYSIS 


6.2. Spearman’s “Тшо-Ғасіот” Solution 

According to Spearman's fundamental theorem,* the necessary and suffi- 
cient conditions for a set of n variables to be describable in terms of just one 
general factor and n unique factors are the vanishing of all tetrads.t The 
relationships between the tetrad conditions and the more parsimonious 
triads have been discussed in 4.3. The '"T'wo-factor" type of pattern may 
be written as follows: 


(6.1) Zi = ajos + a;U; (2-1,2,..., т), 


where Fois the general factor and the U; are the n unique factors, and again, 
asin the case of (2.4), the prime on z; has been dropped for simplicity. When 
the correlations т; (j, k = 1,2, . . . , n) satisfy the tetrad conditions, or the 
equivalent conditions (4.13), the pattern (6.1) may be assumed and the 
coefficients аҙ and а; have to be determined. In the present section formu- 
las will be developed for the computation of the a;; under the assumption 
that the residuals 
Fin = Tik — Tik 


vanish. Then, in the following section, formulas for the ajo will be obtained 
merely on the assumption that the conditions (4.13) are satisfied statisti- 
cally, і.е., within errors of sampling. 

The correlations reproduced from the pattern (6.1) are given by 


ті, = Gjoko ; 
and, under the assumption that the residuals vanish, the observed correla- 
tions may be written 
(6.2) Tik = Te = азак. 
In the remainder of this section the reproduced correlations will be replaced 


by observed correlations. Upon multiplying equation (6.2) by the square 
of the general-factor coefficient for any variable z., this equation becomes 


(6.3) all; = а00;00%0 = (агаа) (Geode) = TejTek , 


and, summing over the correlations, there results 


(6.4) dy X, ra = V reta * is Fry | 


j<k=1 таста j ksze 


* Spearman's theorem was deduced specifically for psychological tests, but it is appli- 
cable to any set of statistical variables. 
1 Formulas for the sampling errors of tetrad differences are developed by Spearman 


and Holzinger, “Note on the Sampling Error of Tetrad Differences," British Journal of 
Psychology, 1925. 
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As in 2.7, the symbol * туь stands for the sum of all the correlations 
j<k=1 

Tir Where j and k each range over the variables 1, 2,..., n but subject 

to the restriction that j is always less than k. In a matrix of correlations, 

this sum is merely the total of all the entries above (or below) the principal 

diagonal. 

Since there is only one common factor, the coefficient of this factor for 
any variable is merely the square root of the communality of the variable. 
Hence formula (6.4) may be written explicitly for the square of the coeffi- 
cient, or the communality, of any variable ze, as follows: 


(тете) к= 1,2,...,т Еее ј< №). 
Lr; J, k -1,2,...,", jk Ae, j < К) 


(6.5) a, = № = 


It will be observed that the diagonal elements of the correlation matrix do 
not enter into formula (6.5). In fact, the formula yields values of the com- 
munalities, which theoretically are the diagonal elements preserving the 
unit rank of the matrix. When the conditions (4.13), which are applied to 
observed correlations, are satisfied, a single factor is postulated. The com- 
puted diagonal elements must also satisfy these conditions in order that 
the rank of the complete correlation matrix be unity. 

It may be illuminating to write out formula (6.5) in detail for a particular 
case. Thus for n variables, the square of the first coefficient is 


aio = 
(6.6) Tiris + Tiaria tb... F Тізгін F 71374 +... таа F Tinin 
Tas F ta Fa: F Та F ra Poat Pon Feret Tas C 


For purposes of computation, formula (6.5) may be expressed as follows :* 


n 2 n 
(5) 5 >: C is 2% f 


6. "Tol ӛзі 
(6.7) a = № = 1 5 diu 
(X - E 
2<Е-1 ізі 


* In the set-theory notations formula (6.7) would be written as follows: 


_ DGeui-heomiseh-Éimuj-eloomik 
= Era; j, k = l,..., m, J <k) Elro; j= 1, . p m j e) 


2 
ао = hi 
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The adaptability of this formula to machine caleulation will be clear from 
the following restatements of the terms in the formula. Let R be the matrix 
of correlations with the elements in the principal diagonal omitted, then 


n 
Жы is the sum of the correlations in column e of Б, 
ізі 

п 
S is the sum of squares of the correlations in column e, 
7=1 

п 
> тук is the sum of all the correlations below the diagonal. 
j<k=1 


After the communalities, and hence the coefficients of the general factor, 
have been obtained, the unique variances can be determined. As pointed 
out in 2.3, the uniqueness for variable е is given by 


a@=1-R?. 


If, in addition, the reliability coefficients of the variables are known, the 
uniqueness may be split into unreliability and specificity by means of formu- 


las (2.11) and (2.12). 


6.3. Computation of a “Тшо-Ғасіо”” Pattern by the Method of Triads 


The method of analysis that will now be described may be employed in 
place of the procedure of the last section. If the conditions for one common 
factor are satisfied by the correlations among n variables, the pattern (6.1) 
may again be assumed. It will become evident that the conditions to be 
checked for a general factor are actually included in the computation of the 
coefficients of that factor and therefore involve no superfluous calculations. 
If one suspects that a general factor will account for the intercorrelations, he 
may proceed to the calculation of a “Two-factor’’ pattern, and in the course 
of this analysis he may determine the adequacy of such a hypothesis. 

The grouping of variables by means of B-coefficients may lead the in- 
vestigator to the hypothesis of just one factor. When the correlations among 
n variables indicate the sufficiency of just one common factor, then the 
communality of any variable z, is given by any one of the following v — 


% " ) triads: 


fas e is fixed 
(6.8) ii. = —— $581: a wap й 


T 2 в 
P <k, Ее 


түн 


турщ 


———— À— 
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Each triad is subject to a small sampling error, so that the » numbers, 
һә, tis, .. . , én—tyn (е omitted), are generally distinct. The most representa- 
tive number Ё?, in the least-square sense, is that which makes the quantity 
Хібз- hSk-12..,nj«k j, ks e)aminimum. The mini- 
mum value is given by the arithmetic mean of the set of numbers f; that is, 


g-lzxQiikel moo j <k, jke) 
(6.9) , 
=} (же, бй без j«h ее). 

In determining the communality of а variable, therefore, the mean of all 
possible triads is used. Since there is only one common factor, formula (6.9) 
also gives the square of the coefficient of that factor for any variable ze. 

The basic assumption which validates formula (6.9), or formula (6.7), is 
that the set of variables can be described in terms of just one common fac- 
tor. In addition, formula (6.7) was deduced under the assumption that the 
residuals were zero. On the other hand, the conditions for just one com- 
mon factor are the equality of all the triads for any variable; so that, strictly 
speaking, all the triads in (6.9) should be equal. The essential difference 
between the method of 6.2 and this section is that in the former case the 
additional assumption is made explicitly in the derivation of the formulas, 
whereas in the latter case no assumptions are made aside from the basic one. 
Of course, the conditions for one general factor are expected to be met only 
within errors of sampling. The method of this section involves the compu- 
tation of individual triads, which may be inspected for statistical equality. 
In the following sections sampling error formulas will be developed which 
may be of use in this connection. 

There is one further assumption which must be made for.the practical 
problem. When the solution for the communality Л? is made by either for- 
mula (6.7) or (6.9), it is tacitly assumed that 


є is fixed 
(6.10) Тат 21 ho bc |,» 


tik - j<k, jke 


Furthermore, for any statistical variable h? < т.ғ, so that 


(6.11) ТЫҢ hs, 

Tik 
where r,z is the reliability coefficient of variable 2: These assumptions will 
usually be satisfied in practical problems, but if a particular triad exceeds 
unity it should be dropped from the evaluation of the communality. 
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6.4. The Bi-factor Method 


The ‘‘Two-factor” pattern is very limited in its application because it 
implies that the rank of the matrix of correlations is unity. Such a simple 
relationship among the correlations is not ordinarily to be expected for а 
large set of variables. The method of analysis as given in 6.2 and 6.3 is very 
useful, however, in the more general bi-factor form of solution. А bi-factor 
pattern of the type (5.8), or one with slight modifications, can be obtained 
for any matrix of positive correlations by applying the preceding methods. 

The procedure is to select appropriate variables out of the total set, which 
taken alone have a matrix of correlations of rank one, and then to apply the 
analysis for variables involving only one common factor. This procedure is 
justified by Theorem 3.5, which states that the dimension of the common- 
factor space is equal to the rank of the correlation matrix. The common- 
factor space in the new situation refers, of course, only to the space contain- 
ing the specified variables. Now these variables are selected from the pat- 
tern plan to involve only one common factor, and so the rank of their ma- 
trix of correlations must be one. 

Тһе complete development of a bi-factor pattern will now be given. Sup- 
pose a set of n variables has been shown to be divisible into m groups б, 
(s = 1,2,..., m) by the method of B-coefficients. If the m groups involve 
pu р, ... , Pm Variables, respectively, the bi-factor type of solution can be 
written as in (5.8). For convenience the pattern* is repeated here: 


2 = Ao + anki 
2 = dao + алі 
Zp, = Apolo + Ap ily 

Zp, +1 = Ap, joo + аф) 

(6.12) в} а * 5% 8 
Zp +r, 7 Aip, p, yo о n (p, +p,)2F 2 
Zn- = Qin-pm+1)0F0 F amonija т 

Zn = Anoo + ақа m 


The problem now is to determine values for the coefficients in pattern (6.12) 
which will reproduce the original correlations, within errors of sampling. 


* See n. *, p. 95. 


| 
| 
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The computation of the coefficients falls into two parts: first, the general- 
factor coefficients, then, the group-factor coefficients. t 

Before developing the methods of obtaining the factor coefficients, the 
geometric interpretation of the bi-factor plan will be briefly reviewed. The 
common-factor space which is used to describe the т variables in (6.12) is 
clearly of (m + 1) dimensions, there being that many factors or axes. It 
may be possible, perhaps, to describe the variables in a space of only m di- 
mensions, but the complexity of each variable would then be greater. There 
is a great deal of geometrical elegance and statistical simplicity in describing 
the variables in the form (6.12). Although the total common-factor space 
is of (m + 1) dimensions, each vector representing a variable lies in a plane, 
that is, each variable measures only the general and one group factor. The 
total configuration is described as a pencil of planes through the Fo axis.* 
When no variable measures more than one group factor, there are no points 
in the planes formed by pairs of group-factor axes.T This geometric inter- 
pretation of the bi-factor configuration gave the clue for further analysis 
into correlated factors, which will be presented in Chapter XI. | $: 

The general-factor coefficients of a bi-factor pattern can be obtained by 
the method of 6.2, applied to appropriate subsets of variables. Any one of 
the triples (e, j, k) of the system ((e, j, К); eeGu, eG, kG, uA 8 з t, 
u, s, t = 1, 2, . . . , m) involves only one common factor. Hence, assuming 
that the residuals vanish and replacing correlations from the pattern by 
observed correlations, there results 


Tjk = ара (еб, Кеб-, s = t) 


according to (5.10). Then, multiplying both sides of this equation by аф, 
and reapplying this equation, the following expression arises: 


(6.13) айға = ааа = (авап)(амаға) = Тее. 


Formula (6.13) gives the value of a2, as computed from the correlations with 
particular variables j and k. To obtain a more reliable evaluation of any 
general-factor coefficient, sum both sides of (6.13) for all values] of j and k 


* Cf. 5.5. 

1 An exception occurs when a variable measures а single group factor and not even 
the general factor. In this case the vector representing the variable actually lies on the 
Eroup-factor reference axis, and hence in the plane of this axis and each of theother group- 
factor axes. 

1 If j and k are merely restricted to be in different groups, and the variables range over 
all groups, each correlation would appear twice, since тук = ть. To avoid this, the indices 
j and k are permitted to range over all groups, namely, jeG, and кеб, for s, = 1, ЖІГІТ 
т, but under the condition s < t. 


# 
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which, together with e, preserve the property of involving only one com- 
mon factor. For any eeG, it follows that 


Z(rara; JeGs, Кеб, 8 <i, з, Ёз и, 5,025 1,2,..., т) 


2 
(6.14) ao = "ye, быз «6, 6, tu, 8 6=1,2,...,m) ` 


This formula gives the square of the general-factor coefficient for any vari- 
able 2,. 

To illustrate the application of formula (6.14) and to clarify the notation, 
a concrete algebraic example will be considered. Let there be fourteen vari- 
ables which, according to the B-coefficient technique, group as follows: 


G = (1,2,3), Gs = (8, 9, 10, 11), 
б. = (4,5, 6,7), Gs = (12, 13, 14). 


Then the bi-factor plan may be formulated as in Table 6.1. The gencral- 


TABLE 6.1 
BI-FACTOR PATTERN PLAN FOR FOURTEEN 
` HYPOTHETICAL VARIABLES 
% Variable Fo n Fi ГА ғ 
ао Ө, Мел ahs vais katt 
азо аз феде mento eint 
аз GU. ЗРЯ РАЖИ з 
аш ЖЕТЕ Жан (Шан 
ао |...... Ul 1-4.--“іьы.. аз» 
О. тры [^ Шо к 
| MENT nn eee, PP 
ШШ” eee eds co mm 45 1...... 
а» | фе кеге ЕГЕТ 
Шыр ЕУР m ‚ша (essc 
шө |. pe ШАҚ, dax seas: 
12,0 |....... ЗЕЕ ТЕТІ Q2, 4 
„лы menn Een Qia, 4 
Xue {Б вага «равна а аша 


factor coefficients can be computed by means of formula (6.14). For ex- 
ample, the first coefficient is given by 


_ (туты) JeGa, Кеб, 5 < t, 5t—2,3,4). N 


2 = 
00 = “Sra; eG, Gas «1 507 2,3,4) ^ D' 


where N and D have been used to denote the numerator and denominator, 
respectively. As it stands, this formula involves the sum of forty correla- 
tions in the denominator and the sum of the same number of paired prod- 
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ucts of correlations in the numerator. The denominator may be written in 
the more conventional (and longer) form, namely, 


7 14 11 М 
р = У У Tik + > > Тұр. 
j=4 k-8 іт8 Ез12 


After some factoring (algebraically), the numerator may be written as 
follows: 


N = Z(ni; Ж) (ris; keG,, s = 3, 4) + (т; јез) (ға; Кеба), 


or more explicitly in the form 


N- (ға + ris + ть + тут) (тв +119 + 71,10 + Tin + Ті,19 + таз 
F rias) + (тіз + то + тло + 71,11) (rae + Tas + 7,4). 


From the values of N and D, the coefficient азо is readily obtained. 
Тһе correlations which are involved in the calculation of аф may be repre- 
sented schematically as in Figure 6.1. The numerator consists of the sum 


© wl o o ajo к юм 


Fio. 6.1 


of appropriate products of correlations appearing in the first column, while 
the correlations entering into the denominator are merely indicated by the 
blocks showing the locations of the variables j and k. The arrangement of 
Work for routine calculation of a numerical problem is described in Appen- 


dix C. 


у, 
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Ап important extension of formula (6.14) should be noted. If the pattern 
plan of a set of variables is of the bi-factor form, but includes a number of 
variables which measure only the general and no group factors, additional 
terms can be included in formula (6.14). For any two such variables, to- 
gether with any other variable e, will involve only one common factor. Тһе 
summations in (6.14) should extend to all such variables j, k; the only re- 
striction being j < k so that no correlation should be used more than once. 
There seems to be no reason for unnecessarily complicating formula (6.14) 
to cope with this case, it being clear how the formula is extended. 

After the general-factor coefficients are computed there remains the prob- 
lem of determining the group-factor coefficients. The first step is to obtain 
the residual correlations with the general factor removed, or the general- 
factor residuals as they will be called hereafter. These are defined by 


(6.15) Та = Тук — ауа (jum 1,9, us Ws 


In general, the table of these residual correlations will be of the form shown 
in Figure 5.1, the values in the rectangles being approximately equal to 
zero. The standard error of a residual correlation will be developed in a 
later section, so that the significance of any entry in this table can be veri- 
fied. 

In the residual-factor space, the n variables can be described by a uni- 
factor pattern, i.e., the bi-factor pattern may be considered as a uni-factor 
pattern with a general factor superimposed. The residual correlations (6.15) 
for each group of variables, taken alone, should have a matrix of rank one 
and hence measure only one common factor. Either of the methods of 6.2 
or 6.3 can now be used for the calculation of the group-factor coefficients. 
Since there will usually be a relatively small number of variables in each 
group, it seems more advisable to use the method of triads of 6.3. By this 
procedure the group-factor coefficient for any variable eG, is given by 


1 A4 . : 
(6.16) ай, = G - қ X =ч, і%6, j<k, j, ke) | 
2 


where, it will be recalled, p, is the number of variables in the group Gs- 
This formula, of course, agrees perfectly with (6.9) when the general-factor 
residuals are considered as the observed correlations and p, is taken to be 
the total number of variables under consideration. 

It may be enlightening to apply formula (6.16) in the evaluation of one 
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group-factor coefficient of Table 6.1. Thus, the coefficient of F, for the 
fourth variable is given by 


A Fans. А : 
MOD і<һ kei) 
2 


3 


1 (Сы Тағат + rate) 
ты Ts: Te: ] ^ 


Тһе complete determination of a bi-factor pattern (6.12) has been shown 
to be possible by means of the formulas (6.14) and (6.16). When all the 
coefficients have been computed, the final residuals can be obtained. These 
ате the residuals after all factors have been removed and may be written 
аз follows: 


(6.17) Fik = Tj — Tja = Pj — араһы. 


If the variables j and k do not belong to the same group, then 7, = fjr. 
Тһе general-factor residuals for variables from different groups, being the 
final residuals, must not be significantly different from zero. At the same 
time, the general-factor residuals for variables within a group must be sig- 
nificantly positive in order for an additional factor to be postulated among 
them. Sampling error formulas for the residuals are given later. 

When it is found that certain residuals between variables of different 
groups are significant, it may be necessary to modify the bi-factor plan 
slightly. The same is true if certain general-factor residuals within а group 
аге practically zero. Some of these modifications will be illustrated later, 
after the sampling errors are considered. 

Тһе method of analysis of this section is adaptable to any matrix of posi- 
tive correlations and yields a very parsimonious solution. The variables are 
probably described in the simplest manner and with complete rigor and 
statistical soundness. Furthermore, with the aid of the sampling error for- 
mulas which are developed in the next three sections, the investigator can 
tell to what degree to factor a set of variables, i.e., determine the number of 
factors that are necessary to reproduce the observed correlations within 
errors of sampling. To aid the reader in applying this method of factoriza- 
tion, а complete outline is presented in Appendix C. There the detailed 
numerical calculations are exhibited on the battery of twenty-four psycho- 
logical tests which has been used before. The calculations of the factor co- 
oe by means of formulas (6.14) and (6.16), have been put in a routine 
orm. 
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6.5. Standard Error of a Factor Coefficient 


The coefficients іп a factor pattern, that is, the correlations of the vari- 
ables with the factors, may vary from sample to sample. Here the term 
“sample” must include the variables as well as the group of individuals. 
Inasmuch as factor analysis is essentially a correlational analysis, the entire 
table of correlations is subject to sampling error due to the sampling of in- 
dividuals and variables from their respective populations. It is evident, 
then, that any formula for the sampling error of a factor coefficient must 
be a function of the number of variables and the number of observations 
of each variable. 

In this section the standard error of a general-factor coefficient will be 
developed. It will be assumed that the coefficient is obtained by the method 
of triads. This assumption is made for the sake of logical rigor because, as 
will be seen, the standard error formula depends upon the triads. Practi- 
cally, however, the formula will also be applicable to coefficients obtained 
by the method of 6.2, since such a coefficient differs very little from one 
evaluated by the method of triads. In addition, the standard error formula 
will be applied to group-factor coefficients after some justification for this 
is made. 

As a first step, it is necessary to determine the standard error of an in- 
dividual triad. Proceeding in the classical manner by taking logarithms of 
both sides of (6.8), there arises 


log tj. = log rej + log rer — log rj. 
Then, differentiating this expression, it becomes 


di da db dc 


(6.18) =+, 


where, for simplicity, the following substitutions have been made: 


(6.19) t= tik, атта, bra, б=т. 


Squaring the expression (6.18) and taking the mean value over the рорша- 
tion gives 


2 2 
"mE cb Табь Cafe 040, 
== жа +2 Tab Tac 2). 


ot 


ab ac be 


The purpose now is to reduce formula (6.20) to an expression in the origi- 
nal correlations a, b, and c. This is accomplished by means of, first, the 
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well-known formula for the standard error of a correlation coefficient, 
namely, 
2-01-09: 

(6.21) 9. = oum ; 
where N is the number of observations in the sample; and, second, the Pear- 
son-Filon equations for the correlation between correlation coefficients. The 
form of these equations for the case where an index is repeated on the two 
correlations may be written* 

ab(1 — а? — b? — c? + 2abc) + 
(6.22) Tay = C 2ü — a) — b5 


In the derivation of these formulas it is assumed that the variables are dis- 
tributed normally in the population. Substituting expressions of the type 
(6.21) and (6.22) for the o’s and "іп (6.20), this equation becomes 


= qi — pr? gine —aey(1 = b 
ха = e! =£) «( 22 +(? “) 42001 zu ) 


с 


2b(1 — а?)(1 — c) 42: = Dd — d) 
ac с 


-а-ғ-в-ежте). 


By algebraic reductions this expression can be put in the form 


ZG 5 ab? а% а ab? ab? 
ata “ши сш abor Sr zn 


(6.23) No? = 2 n 


This formula gives the standard error of a particular triad. Formula (6.23) 3 
may also be written as follows: 


(624) No? = (а - 2 (4 Ж 5) +4420 - 00-204 1 ; 


Which is а simple form for computation. 
In the above development only the variation in the sample of individuals 
was considered. When the sampling error of а communality or a factor 


* Karl Pearson and L. N. С. Filon, “Оп the Probable Errors of Frequency Constants 
and on the Influence of Random Selection on Variation and Correlation," Phil. Trans. 


Roy. Soc., 1898, p. 263. 
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coefficient is desired, however, the variation in the sample of variables must 
be taken into account. Inasmuch as no method for dealing with the simul- 
taneous variation of individuals and variables is known, approximative 
methods are used. The general procedure, in making allowance for the vari- 
ation of variables, is to employ averaging methods. 

In obtaining the standard error of а communality, all variables will be 
considered by taking the mean of all possible triads. In the case of only one 
common factor, of course, each coefficient is merely the square root of the 
communality, that is, a.o = МЇ (е = 1,2,...,m). Hence the standard 
error of the coefficient can be readily deduced from the standard error of 
the communality by means of the formula* 


(6.25) 02, = in ай. 


According to the foregoing argument for developing an approximate for- 
mula for ca, it would be necessary to obtain the mean value of (6.24) for 
all possible triads and then apply (6.25). It appears to be somewhat simpler, 
however, to obtain the standard error of a particular evaluation of а coeffi- 
cient by means of (6.24) and then to take the mean value over all possible 
evaluations. Thus the value of the coefficient from the triad t is Vt, and 


(6.26) а; = 261. 


The formula for c*;; in terms of the original correlations then becomes 


- a,b  j(aV b\? ab ab ab\? 
(6.27) АМ = 2 eoe (тү - 2(2) +@-5®+4( ): 


с Bi 


There now remains the problem to take the mean value of (6.27) over all 
possible triads. 

The terms in the right-hand member of (6.27) group themselves into 
five distinct types of nonlinear functions of two or three of the correlations 
a, b, c. The means of such functions can, in general, only be expressed in 
terms of the means, standard deviations, and correlations of the original 
variables (i.e., the correlations a, b, c) to a first approximation. Theimplicit 
assumption is that the deviations are small compared with the means of 


* For the general theorem relating the standard error of a function to the standard 
errors of its variables see Appen. B.T. 
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these variables. Designating the mean over v values by a bar, the means of 
the various types of functions in (6.27) may then be tabulated as follows:* 


zs jo 
3 


Sha doeet) 
ыды ы 
(6.28) 2y = a (1 Ee 5 + ге + E _ мылы Е аав 302 
(59-ы edt) 
(2) E a (1 % ‚ы е Заты -" Зал " ы 


where the standard deviations and correlations are over the observed > 
values of the original correlations. In the derivation of these formulas it is 
assumed that the ratio of the deviation of a, b, or c from its respective mean 
to that mean is so small that third- and higher-degree powers can be neg- 


lected. 


The mean value of oł; over all possible triads will be written c2, to in- 
dicate the passage from an individual evaluation of a factor coefficient to 
the final value based on all > triads. Then, substituting the values from 
(6.28), and other expressions like these, for the mean values of the ratios . 


in (6.27) gives 


teo 


sN. = 
L' 


ё 
ad. a^ 602, 648 5ab е) 
T UP на КОЮ ТЕ GENE. 
И сабьаь | a b ab , 5аь 164% 
ioe е е = ш uir e к 
ab с ae ё с i 
NU (а b = sa T Sah 5ab | | 
ас be ас С 6 
nu (3 diia. 8и за | Ба) юа) А 
bc be ас ё c с 


* For the method of deriving these formulas see Appen. B.8. 
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Formula (6.29) gives the complete expression for c2, to the degree of 
approximation indicated by (6.28). The complete formula, however, seems 
to be too long for practical purposes. It has been checked empirically that 
the first bracket gives а very good approximation to the complete expres- 
sion. The indication seems to be that the second bracket contributes & 
small positive quantity to the right-hand member. If the function of the 
correlations in the entire right-hand member is designated by f(r) and that 
in the first bracket by ¢(r), then the preceding observations may be stated 
as follows: 

f(r) > Ф(ғ), 


or 
(6.30) f(r) = Ф(0) + 0°, 


where à? is а small positive quantity. 
Now the desired standard error is Ta, so that it is necessary to express 
c£, in terms of 62,,. It is a well-known fact that 


eS a, 


where the standard deviations аге for any variables. The last expression 
may also be put in the form 


(6.31) с = 0 е, 


where e? is а small positive quantity. Making the substitutions (6.30) and 


_ (6.31) in (6.29), there arises 


‚уо $0, 9 
(6.32) a, = IS. 


The errors tend to compensate for one another, and the final formula is 
taken to be 


ЕЕ 1 
6.38 2.7 —— 
(6.33) 0а = gy 900 - 
Written out fully, and dropping the scoring over the c, this formula becomes 


1 ау 2a 20 аЬ 5а 1°? 
6.84) cà, = (2 ‹ _2@_ аў Sab , 44% 
(6.34) vi, AN uu e ете ё P ; 


This represents the best approximation to the required formula for the 
standard error of a general-factor coefficient. 

To clarify the meaning of the terms іп (6.34), suppose the number of 
variables in the set to be five and that the standard error of the first coeffi- 
cient of the general factor is required. The coefficient is denoted by aio and 
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5-1 


is evaluated from » — ( 2 


) — 6 different triads. Its solution, according 
to (6.9), is given by 
ai, = Ж = ӘЛЕН 1Е-2,3,45, ict). 


Tjk 


The parameters defined in (6.19) are, for this case, 


іші, а = ту, bn, C = Tjk 
(j, k = 2, 3, 4, 5, j«k). 


The meaning of the terms in (6.34) will then be evident from Table 6.2. 


TABLE 6.2 

ab 

jk a-rnj b =rik c=rjk пье 
рл. ааа» т: таз Tos із 
2 әлемет т Tu тщ tu 
25 су | me Tis Tas los 
Bá. aues] Ті Tu ти tu 
‚ЖӨ тз Tis Tss іш 
бекас ак ти Tis ть las 
Mean ü b ё ajo 


Here all the correlations leading to the averages 4, b, 6 are written down 
for clarity. In an actual problem, however, considerable time can be saved _ 
by omitting these three columns. The stub, giving the indices, should be 
retained to keep track of the variables, but this can be followed immediately 
by the column of triads. The appropriate correlations for computing d, 5, €, 
and the individual triads can be read off directly from the matrix of corre- 
lations. A numerical example, illustrating the computation of (6.34), will 
be given later. 

Before leaving the present discussion, it may be well to point out another 
application of the standard error formula (6.34). This formula was devel- 
oped on the assumption that the general-factor coefficients were computed 
by means of formula (6.9), i.e., the method of triads. As already remarked, 
formula (6.34) applies equally well when the general-factor coefficients are 
computed by the classical method indicated in 6.2. Now it would be very 
convenient if this standard error formula also applied to group-factor coeffi- 
cients. Such coefficients are obtained by the method of triads, as may be 
seen from formula (6.16). The essential distinction between (6.9) and (6.16) 
is that every correlation in the former is replaced by a general-factor residual 
in the latter. When the development of the standard error of a group-factor 
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coefficient was attempted, employing triads of general-factor residuals, a 
very unwieldy formula resulted. A very good approximation to the stand- 
ard error of a group-factor coefficient, however, is given by formula (6.34) 
in which the observed correlations are replaced by general-factor residuals. 


6.6. Standard Error of a Residual 


It will be recalled that a residual is defined by 7;; = rj; = ту, where 
тук is an observed correlation and rj, is the corresponding correlation re- 
produced from a factor pattern. For the case of just one common factor 
The = араке and Fj), = Tjk = азак. The total differential for 7;, is 


Фар = drj, — азак — аша)», 
which, upon squaring and taking the mean value over the population, gives 
2 = a 2 42 2 42 
(6.35) Op = Оту + аса, + (9025, + 2[а}оа коса j, Tapa ар 
185 


— азс, ца да, - 010074, Sai дары? 


At the present time по formulas are known which express the correla- 
tions in the bracket in terms of the original variables, ajo, а, and гу. 
Hence the following expression 


(6.36) б = заб + ао, + ai 


ajo 


is the best approximation to the standard error of a residual, for one factor 
removed. The standard errors involved in formula (6.36) may be computed 
by means of (6.21) and (6.34). 

When two variables, z; and z;, involve two uncorrelated common fac- 
tors, Fo and Fi, their residual correlation is given by 


(6.37) Pik = Tik — Qjodko — адан. 


The result of differentiating and squaring the expression (6.37) and then 
taking the mean for the population is 


ЖЕТІ: 222 2 2 2 
Trip = Orin + афо + афо + ао, + айо, 
+ 2(ауоако® кос jo aja F азса са, „Са Тама 
(6.38) F азана, „оа Ға, aj, F ko j1Faj,Fay,7e jit 


t (400/106, 8a; Таза + алака, ба Тала 
T 4100: Das қак, — 000510, Trigo jo 


i — GAG r аба, Туда — Tra Das та). 
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The best approximation for the standard error of the residual (6.37) is 
given by - 
(6.39) с? 


2 2 62 222 222 о 2 
Fik = Cf 1 оса, 1 (004, + аўса, 7 аца, · 


The foregoing results can easily be extended to the general case. If two 
variables have a general and m group factors in common, they may be de- 
scribed as follows: 


2 = а,” + anal) Баве +... + pr 2и 
гь = ajoF o + аһа + аеЕ +... + dimlm 


The final residual correlation is 
m 

(6.40) Tj = Tjk — SY anan А 
8-0 


Then, as before, it can be shown that a good approximation to the standard 
error of this residual is given by 


m 


ә b] ә о 
(6.41) ёа = ola +>, (ао, + ао). 
s=0 


Ostensibly, formula (6.41) gives the standard error of any residual after 
any number of factors have been removed, without any restrictions as to 
the type of factor pattern or method of computation of the coefficients. 
Such restrictions are implied, however, in the practical application of this 
formula, The formula involves the standard errors of the factor coefficients, 
which are not known for any type pattern but the two-factor and bi-factor. 
After further approximations, in the next section, a formula for the standard 
error of a residual will be obtained which does not involve the standard 
errors of the coefficients explicitly. In that form the formula may be ap- 
plied as an approximation to other types of factor solutions, for which the 
sampling errors of the coefficients are not known. 


6.7. Further Approximations to the Sampling Error Formulas 


Other approximations to the sampling errors of a factor coefficient or a 
residual may be obtained by means of additional arbitrary, but reasonable, 
assumptions. For example, if a set of correlations is more or less homogene- 
ous, one may assume a constant value for any one of the correlations. Thus 
the correlations used in the evaluation of any factor coefficient may be set 
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equal to their mean value, say p, yielding a2; = р. Under this hypothesis, 
formula (6.34) for the standard error of a coefficient reduces to 


(6.42) a= gp Ê- 2- 50ta), 


which applies to any coefficient. The index a should not be confused with 
that defined in (6.19). Since this formula applies to all а.о (e = 1, 2; asm 
n), the subscripts have been dropped. 

Then, employing the value (6.42) for the standard error of a coefficient 


in (6.36) but letting the other elements retain their individual identities, 
that formula becomes 


(1 = rh)? | а + ai (3 
WS 1 1 9 = 2 
(6.43) [M N + iN р 2 — 5p + 4р? |. 
For a rough estimate of the standard error of any residual, with one factor 
removed, an even simpler formula can be used. In place of the individual 
correlation гу the average correlation p can be used, and, setting аз = 
аһ = p, formula (6.43) reduces to 


2201-95-80 + 207 
(6.44) а= 2036 beta 


where the indices have been dropped from the residual correlation because 
this formula applies to any residual. 

Approximate formulas for the standard error of a residual, with two or 
more factors removed, can be obtained similarly. Substituting (6.42) for 
615, 0%.) and similar expressions for c2; , aap, into (6.39), and setting тук = P: 
aj, = аз = р, and aj, = а = р, 


„ _ (1— 0)(5 + 8р + 202) , m (3 
(645) 4) Ж + fy (2 = 2-50 +40), 
where 
p = average correlation used in computing Fo coefficients, 


рі = average residual correlation used in computing F, coefficients. 


In general, the standard error of a residual with (m + 1) factors removed 
is given by 


1 — p)*(5+ 8p + 2p 5 
вав) = СОВА LN do so db e 22D, 
8-1 
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where p, is the average residual correlation used in computing the F, coeffi- 
cients. 

Any one of the formulas (6.42) to (6.46) must be used with great caution 
and with a full realization of the underlying assumptions. Most important 
of all, one must check whether the assumption of setting all observed corre- 
lations equal to their mean is a tenable one. This assumption, of course, is 
over and above those assumptions made to get more or less practical formu- 
las like (6.34) and (6.41). Fortunately, the general direction of the dis- 
crepancies in using any of the approximate sampling error formulas is fairly 
well known. It has been verified empirically that at each stage of approxi- 
mation the values obtained for the sampling errors generally become small- 
er. Аз has already been pointed out, the standard error of a coefficient as 
given by (6.34) is usually smaller than that given by (6.29). Again, if (6.34) 
is approximated by (6.42) the value generally becomes a little smaller. As a 
consequence of this, the standard error of a residual according to formula 
(6.46) is generally somewhat smaller than that obtained by formula (6.41), 
unless the original correlation is much larger than the average employed 
in the former formula. By knowing the general direction of the discrepancies 
in the approximations, the investigator can make due allowance in setting а 
level of significance.* 

With the foregoing limitations clearly in mind, it may still be very desir- 
able to use formula (6.42) and the various formulas for the standard error 
of a residual. For this purpose some of these formulas have been put in 
tabular form and are presented in Appendix Н. The standard error of а 
coefficient for an average correlation from р = .10 to p = .75 and for 
samples from № = 20 (о N = 500 is presented in Table Н.1. For the same 
range of values of p and N Table H. 2 gives the standard error of a residual 
for one factor removed. Only one supplementary table is necessary, and, 
with very little computation, any value of (6.46) can beobtained. Table H.3 
gives the values of 

(G — pe — р: + 2р). 


Thus, in general, the standard error of a residual with (m + 1) factors re- 
moved is obtained as follows: square the entry in Table Н.2, for the par- 
ticular p and N, and add to this 1/N times the sum of the m entries of 
Table Н.З, corresponding to the m values of р,; then the square root of this 
sum is the required standard error of the residual. 

* For example, if a particular residual is just twice its standard error (as given by one 
of the approximate formulas), it can safely be said that this residual is probably insignifi- 
cantly different from zero. The argument is that the standard error is probably а little 
larger, and the ratio a little less, than two. For such investigation, then, the level of sig- 
nificance should be taken to be at least three times the standard error. 
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The statistical tables of Appendix H, or the more exact formulas of 6.5 
and 6.6, may be used in determining the standard error of a factor coefficient 
or a residual correlation when such statistics refer to a bi-factor solution. 
For other types of factor solutions, which were presented formally in Chap- 
ter V and which will be described further in later chapters, the sampling 
error formulas do not apply. Since no sampling error formulas are known 
for other types of solutions, the formulas which have been developed for 
the bi-factor solution may be used, in such cases, as first approximations. 


6.8. Interpretation of Standard Errors 


The statement that a statistical constant (e.g., a factor coefficient or 
residual) has a value z with a standard error e; means that, if the constant 
be determined many times from random samples of relevant material, the 
observed value х obtained in a sample will differ in absolute value from 2 by 
less than о: in approximately 68 per cent of the number of samples. This 
conclusion is based on the assumption that the sampled values of z are dis- 
tributed in accord with the normal law; the value e. ceases to have the same 
significance if the distribution is not normal. 

The equation of the normal distribution of x may be put in the form 


(6.47) y = == е?“ 


where the origin is at the mean and the total area under the curve is unity. 
The values of z = x/o, or the number of standard deviations from the 
mean, аге called deviates, and ža will be used to denote the area from the 
mean to such a deviate. 'Then 


E 1/2)? z 
(6.48) a дее T u pe ale FE TM 
0 2r Jo 
The area $a gives the frequency of occurrence, or the probability of occur- 
rence, in the range from the mean to z/c. Table H.4 gives this total fre- 
quency, or probability integral, for values of т/е from 0 to 4 in intervals 
of .02. 

Frequently the question is to determine what fraction of the total popu- 
lation has a larger deviation than +х/е; or, in other words, what is the 
probability that a statistical constant so distributed, and chosen at random, 
will exceed a given deviation in absolute value. This probability is given by 
the areas in the tail pieces beyond +2/c, which may be written 


(6.49) Pel 


ħħ ———— 
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If the probability of exceeding +2/c is required, it is, of course, given 
by 4P. 

The problem of greatest interest is to determine the significance of sta- 
tistics obtained from a given sample. A statistic is an appropriate function 
of the values of a variable given by a sample and used to estimate the corre- 
sponding population parameter, which is a hypothetical quantity helping 
to specify the mathematical form of distribution in the universe. Now an 
observed variance s? may be used as an estimate of the parameter c? of the 
distribution of x, or, better still, the mean value of s* from repeated samples 
may be employed. The variance c? refers to deviations from 2, whereas 
any s? refers to deviations from an z obtained in a particular sample. The 
value 2 may be considered as an arbitrary origin for a given sample, and 
the sum of the squares of deviates from this arbitrary point will generally 
be greater than the sum of the squares of deviates from the mean of the 
sample (by the least-square property of the arithmetic mean). Hence it is 
to be anticipated that the mean of the values of s* will be less than o°. The 
extent of this inequality is given by* 


CN Ed 
B(spe a= е 


where E has been used to denote the expected value from repeated samples. 
It is advisable, especially when dealing with small samples, to use the un- 
biased estimate of the population parameter, which is defined by 

a= 
If N is large, the coefficient in the right-hand member approaches unity, 
and it is not invalid, to any appreciable extent, to use s in place of the un- 
biased estimate. 

When the “true” value of a parameter 2 is known, and when the distribu- 
tion of this statistic in successive samples follows the normal law, the prob- 
ability of obtaining a deviation from the “true” value less than any given 
multiple k of the standard error of the parameter іп question is given by 
the area under the normal curve inside the range + kez. For example, if 
the observed value of a general-factor residual (say, х = .255) and its 
standard error (c, = .117) were the true" values, then, referring to Table 
H.4 for the area, 3a = .3413 corresponding to one deviate, it may be con- 
cluded that the probability is a = .6826 that in further samples from rele- 


* See Dunham Jackson, “Mathematical Principles in the Theory of Small Samples," 
American Mathematical Monthly, 1935, рр. 344-64. 
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vant material the value of the residual obtained will be within the range 
.255 + .117. Similarly, it may be argued that the probability is a = .9544, 
or that in about 95 per cent of the samples the value obtained will be within 
the range .255 + 2(.117). These results may be stated in another manner. 
Under the same assumptions the residual obtained will differ from .255 by 
more than .117 in absolute value (i.e., it will lie outside the range of .138 
to .372) in 32 per cent of the cases; and it will differ from .255 by more than 
.234 = 2(.117) units in only 5 per cent of the cases. These are the percent- 
ages (to the nearest integer) of the total area under the normal curve outside 
of the respective ranges +o and + 2с. Obviously, the greater the range 
adopted, the less likely it is that the sampled value will fall outside of it. 

When only the standard error, but not the parameter, is accurately 
known, the table of the probability integral may still be used to answer the 
question whether the observed parameter differs significantly from any 
given hypothetical value. This is usually the more important question. 
Thus, in the present illustration, it is not so important to know how close 
the value .255 is to the “true” value of the residual, but whether the ob- 
served value is significantly different from zero. Assuming that the stand- 
ard error .117 is the true value of that parameter, it is found that the devia- 
tion 0-.255 is 2.18 times the standard error. The area under the normal 
curve to the left and to the right of the range +2.18 is 


Р = 1 — a = 1 — 2(.4854) = .0292, 


where 3a = .4854 is obtained from Table H.4 for the ratio .255/.117 = 
2.18. This means that if the residual be determined many times from ran- 
dom samples of relevant material, for which the true value of the residual 
is zero, the value .255 would be exceeded in absolute value in 2.92 per cent 
of the cases. Even more important, however, is to test the hypothesis that 
the value obtained for the residual is significantly positive. In this case only 
the area to the right of 2.18c should be computed. The corresponding 
probability is 
4P = $(.0202) = .0146, 


which means that in sampling the residual, from an assumed true value of 
zero, the value .255 would be exceeded positively in only 1.46 per cent of 
thetime. Consequently, one may conclude that the true value of the residu- 
al is different from zero or, for the second hypothesis, that the residual .255 
is significantly positive.* By the same procedure the significance of the de- 
parture of the residual from any other hypothetical value could be tested. 

* Of course, if there is sufficient evidence, on other than pure statistical grounds, for 


the validity of the hypothesis that the true value is zero, then it would not be rejected. 
One might argue, however, that the particular sample was а very unusual one. 
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In drawing conclusions, the investigator may be more or less exacting in 
the smallness of the probability that he would require. *It is usual and 
convenient for experimenters to take 5 per cent as a standard level of sig- 
nificance, in the sense that they are prepared to ignore all results which 
fail to reach this standard, and, by this means, to eliminate from further 
discussion the greater part of the fluctuations whieh chance causes have 
introduced into their experimental results."* With this standard for the 
level of significance, an investigator would attribute to “риге chance" re- 
sults having probabilities up to 1 in 20. In the preceding example the odds 
against the conclusions on a pure random sampling chance are .9708 to 
.0292 or 33 to 1 in the first case, and .9854 to .0146 or 67 to 1 in the second 
case. It is thus seen that the conclusions were drawn in accordance with 
common practice. 

Although the foregoing procedure cannot, strictly speaking, be applied 
to parameters of factor analysis because the standard errors are not accu- 
rately known, it may be used when N is large. In factor analysis the samples 
are usually large, and for this reason it is suggested that the ratio of the de- 
viation (of an observed statistic from an assumed value) to the standard 
error be referred to a normal probability scale. It should be emphasized 
that, when the "true" value of the parameter is taken to be zero and the 
preceding test is applied, there are really two hypotheses being tested: first, 
that zero is the true value and, second, that the parent distribution from 
which the sample is drawn is a normal one. An extremely high value of the 
ratio might then suggest either that zero is not the "true" value of the 
parameter or that the distribution is not normal.t The conclusion would 
be that the observed value of the parameter is significantly different from 
zero or that the assumption of a normal parent population was unjustified. 

To guard against drawing erroneous conclusions, the investigator must 
supplement his purely statistical tests of significance with all the theoretical 
and factual knowledge at his disposal. Thus, the general-factor residuals 
among a set of variables, which were shown to belong together by the 
B-coefficients, may all turn out to be positive but not very significantly 
different from zero according to the purely statistical tests of significance. 
In this case, although it might be argued that the residuals are insignificant- 
ly different from zero (with perhaps probabilities of only up to 1 in 20 
against pure random sampling chance), such a hypothesis would be re- 
jected. The very consistency in positive values would lead the investigator 
to doubt that every one of these residuals was really a sampling fluctuation 


* R. A. Fisher, The Design of Experiments, 1935, pp. 15-16. 


1 The assumption of normality is made, for convenience, in practically all sampling 


problems. 
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from zero, always in the same direction! A group factor would be postulated 
in such a case, and the final residuals for this set of variables would be ex- 
pected to vary positively and negatively around zero. 

'The standard errors derived in this chapter must be used with great 
caution in drawing probable inferences, for they are, at best, only approxi- 
mations to the true values. Furthermore, the tests of significance imply а 
normal parent population from which the sample is drawn. The form of 
distribution of a triad, a factor coefficient, or a residual in successive samples 
is not known; for, even if the correlations were normally distributed, the 
same could not be said, a priori, of these nonlinear functions of the correla- 
tions. It is evident, then, that the standard errors of factor analysis do not 
have the same heuristic properties that they are presumed to have in the 
natural sciences. The standard errors enable the investigator to answer the 
question, “If the observations constituting the sample are drawn at random 
from a normally distributed universe, and if the true value of the parameter 
is 2, what are the odds against the observed value х having arisen as а result 
of the fluctuations of sampling?" In this role the standard errors become 
one of the measures of goodness of fit and, as such, are extremely useful 
and often indispensable. 


6.9. Verification and Modification of the Bi-factor Pattern 


Тһе sampling error formulas may be employed for the same purposes as 
such formulas are used in other branches of statistics. Thus, the significance 
of any factor coefficient or residual may be tested. Again, if two factor pat- 
terns, based upon two samples of individuals, are obtained, the difference 
between any two factor weights may be tested for significance. Some other 
applications of the sampling error formulas suggest themselves for factor 
analysis. Formula (6.24), for the standard error of a triad, may be used to 
justify the conditions for one common factor. This formula may also be 
applied at the second stage of analysis, i.e., when the general factor has been 


removed and a certain set of general-factor residuals are considered as 
measuring one group factor. 


Тһе refined check for one common factor among a set of correlations may 
be obviated, however, by employing the B-coefficients, dropping all triads 
which exceed unity, and observing the table of intercorrelations for in- 
significant correlations for an entire row and column. If several variables 
apparently form a group, but the set of general-factor residuals for that 
group includes a row and column of insignificant values, the variable with 
these residuals should be dropped from the group and should not be used to 
measure that group factor. Such a variable then measures only the general 
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factor, and its residuals with all other variables are then final residuals. On 
the other hand, if some of the final residuals turn out to be significantly 
different from zero, further factorization is indicated. Variables producing 
such residuals have a factor in common, and the pattern plan must be modi- 
fied to inelude these changes. If a new factor is indicated for a subset of 
variables, then the preceding group-factor coefficients for these variables 
must be modified, for the hypothesis that these variables had involved only 
that one group factor is then invalidated. In computing the common-factor 
variance, beyond the general factor, the variables must be assumed to have 
a correlation matrix of rank two rather than rank one. The important thing 
is to formulate the new pattern plan; once this is done, the actual computa- 
tion of the factor coefficients is a simple matter. First, triads are selected so 
that no two variables measure more than one common factor. Then formu- 
la (6.9), with the appropriate triads, may be used to calculate the coeffi- 
cients. If the overlapping between variables is so great that appropriate 
triads cannot be selected, then a formula of type (4.22) may be used and 
the coefficients determined as in 4.7. 

Тһе procedures for modification suggested in the last paragraph may 
seem to be somewhat subjective. Nevertheless, it can be done very quickly 
and checked by the more exact sampling error formulas. The significance 
of a coefficient or residual may be determined from the statistical tables in 
Appendix Н; or, in particular instances, formulas (6.34) and (6.41) can be 
applied. Not only particular residuals but the arrangement of the entire 
set for a variable, or several variables, may give a clue for modification of 
the factor pattern. For example, if there is one large residual while all 
others are insignificant for а given variable, a doublet (faetor through only 
two variables) may be postulated between the two variables producing 
that residual. 

It should be noted from Table 4.2 that, when only two variables are as- 
sumed to measure а factor, their communalities can have an infinite number 
of solutions, so that it requires at least three variables to determine the 
factor weights uniquely. The two variables may therefore be considered as 
indicating a potential factor, which might have appeared if more measures 
of that factor were present. In determining the doublet weights for the 
given variables, one standard deviation of the residual may be arbitrarily 
selected as chance error and the remainder divided equally between the 
two variables. Thus, if the one large general-factor residual is between 
2; and гь, and the doublet is denoted by D, and its coefficients by ал, dia, 
then 


(6.50) di = da = М? — 0+. 
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For the example of the last section, if the significant residual 7;, = .255 
conformed to the conditions for postulating a doublet, the coefficients of 
variables j and k for this doublet would be 


dj = да = V.255 117 = У 138 = 371. 


The final residual would be 7;; = .117, which, of course, is insignificantly 
different from zero. 

Special attention should be paid to the analysis after the general factor 
has been removed. Slight modifications in the grouping of variables and 
new factors between several variables will not change the general-factor 
coefficients very much. Each of the general-factor coefficients is based on а 
large number of correlations so that a few changes in the pattern plan have 
little effect on them. A few discordant triads in the calculation of a group- 
factor coefficient, however, may greatly affect its value. For this reason, no 
triad should be used which exceeds unity. Also, no triad should be used 
which involves insignificant general-factor residuals. One other method 
might be used in the verification of variables measuring a group factor. The 
method of B-coefficients may be applied to general-factor residuals for pur- 
poses of verifying the elimination of certain doubtful variables from the 
original group. The B-coefficients for combinations of two or three vari- 
ables, including the doubtful one, may be calculated and their magnitude 
inspected for the degree of belonging together. 

The preceding discussion is mainly concerned with modifications of the 
bi-factor pattern when the residuals seem to indicate that higher com- 
plexities are required for some variables. By verifying the significance of 
particular factor coefficients, the complexity of a variable may be reduced. 
Thus one or two general-factor coefficients might be of the order of their 
standard errors and in further sampling they might be zero. In the modified 
bi-factor pattern these variables would be dropped from measuring the gen- 
eral factor. Hence the final solution would not involve a general factor. 
The blankness in a bi-factor pattern can similarly be justified by means of 
the standard error of a factor coefficient, for, if a coefficient were computed 
wherever there is a blank, its value would be insignificantly different from 
zero. 

An illustration of one type of modification will now be given. Table 6.3 
contains the observed intercorrelations of five physical variables, taken 
from a total set of seventeen and based upon N = 305 fifteen-year-old 
girls.* The complete analysis of the seventeen variables is of no concern 


* These data are taken from Frances Mullen, "Factors in the Growth of Girls Seven 
to Seventeen Years of Age," 1939. 
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here, the particular portion having been selected to illustrate а point. A 
complete bi-factor solution will be exhibited in the next section. The pres- 
ent example of five variables is introduced to clarify some of the preceding 
ideas on the application of sampling error formulas and modification of а 
bi-factor solution. 
TABLE 6.3 
INTERCORRELATIONS OF FIVE PHYSICAL VARIABLES 


Variable 1 2 3 4 5 


Su o tor 


In the total set of seventeen variables the five variables of Table 6.3 
were found to belong together by the method of B-coefficients. The bi-factor 
pattern plan was then assumed to contain а general physical (or growth) 
factor, say Fo, and a group factor, say Еу, through these five variables. The 
general-factor coefficients were caleulated by the method outlined in Ap- 
pendix C, and for the given five variables these coefficients are aio = .691, 
а» = .591, азо = .581, as = .598, and а = .674. Тһе general-factor re- 
sidual correlations among the five variables are presented in Table 6.4. 


TABLE 6.4 


GENERAL FACTOR RESIDUALS AMONG 
FIVE VARIABLES 


Variable 


S es 

.538 
473 |. 
‚099 | .102 | .048 |...... 


The immediate problem is to determine whether the original hypothesis 
that the five tests measure a single group factor is warranted from the na- 
ture of the residual correlations in Table 6.4. A general inspection of Table 
6.4 seems to indicate that variable 5 does not exactly fall in line with the 
other four. It seems quite likely that variable 5 has additional linkage with 
variable 1, beyond the general factor, but that no further linkage with the 
other three variables is necessary. In other words, the general-factor residu- 
als of 5 with 2, 3, and 4 might very well be final residuals. This conclusion 
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will be checked by various methods in order to exhibit the procedures for 
modification of the pattern plan. 

First, the method of B-coefficients will be applied to the general-factor 
residuals to verify the elimination of the doubtful variable 5 from the origi- 
nal group. The B-cocfficients for combinations of 5 with the other variables 
are presented in Table 6.5, where the notation employed is that of Chap- 
ter II. The value B(1, 5) — 107 indicates that the variables 1 and 5 belong 
together to just about the same extent to which they belong with the other 
three variables. All the remaining B-cocfficients of 5 with the other vari- 
ables are under 100 and clearly indicate that 5 does not belong in the group. 


TABLE 6.5 
B-COEFFICIENTS BASED ON GENERAL-FACTOR RESIDUALS 


| 
| opu , Я ч _ 200(n—p)8 
и p | 5 | 200(n — p) | T (p -1)T Blu) = (p-DT 

(1,5). СА 2274 600 1.537 1.537 107 
(2,5). | 2 099 600 1.873 1.873 32 
(3,5) 2 .102 600 1.817 1.817 34 
+ Ж RUM 2 ‚048 600 1.848 1.848 16 
(Te 8) ЖИР 3 -811 400 2011 4.022 81 
(1,8,9): ++ 3 -780 400 2.023 4 046 77 
(Ds AsO) у, Ж ‚7168 400 1.970 3.940 78 
(9.959). « 3 ‚739 400 2.091 4.182 71 
(2,4,5).. | 8 ‚620 400 2.252 1 504 55 
(3, 4, 5).... | $9 604 400 2.234 1.468 54 
(2,3,4,5). 4 1.714 200 1.562 4.686 78 
(1,2,3, 4).. 4 2.753 200 -523 1.569 164 


Тһе last entry, B(1, 2, 3, 4) = 164, is included іп Table 6.5 for contrast. 
"There is no doubt that variables 1, 2, 3, and 4 belong together, while vari- 
able 5 probably does not belong with this set. 

Тһе modification of the pattern plan implied by the preceding analysis 
15 to drop variable 5 from measuring the group factor Fy. The entries in the 
last row of Table 6.4 would then be final residuals. Now the final residuals 
must be zero except for errors of sampling, so that these residuals should be 
tested for significance. For purposes of comparison, the standard errors of 
these residuals have been computed by formula (6.36) and also from Table 
H.2 and are listed in Table 6.6 together with the ratios of the residuals to 
their standard errors and the probabilities that, in random sampling, the 
observed deviations of the residuals from zero would be exceeded in absolute 
value. The probabilities that the observed residuals would be exceeded in 
the positive direction only are just one-half the respective probabilities 
given in the table. 

In applying formula (6.36), the standard errors of the general-factor co- 
efficients are required. For the present example the approximate value oa = 
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1066 from Table H.1, corresponding to an average correlation p — .355* 
and N = 305, was employed. Another numerical comparison of standard 
errors, in which the standard error of a coefficient is computed by means of 
(6.34), will be given in the following section. The two sets of standard errors 
in Table 6.6 are very little different. 
The probability P for the residual 715 
and clearly indicates that in sampling, 
observed value would be exceeded in less than lp 


TABLE 6.6 
STANDARD ERRORS OF RESIDUALS 


= .274 is zero to four decimal places 
assuming a true value of zero, the 
er cent of the cases. It 


Bv Forura (6.36) From Taste H.2 


VARIABLE Resmo- 
AL 
j tjs oF tor |Ре1-а| oF | Tos | БыТ-а 
пее 274| .069| 3.97 | .0000 | .074 3.70 | .0002 
Sa ханын ‘909 | :073 | 1.36 | .1788 | .074 | 1.94 | .1802 
Brox, quine "999 | 2073 | 1.40 | -1616 | .074 | 1.38 | .1676 
Sou iy ust .048 .075 0.64 | .5222 .074 0.65 | .5092 


ded that the true value of this residual is different 
es for the remaining three residuals are each 
d level of significance recommended in the last 
Section, and so the deviations of these observed values from zero may be 
attributed to chance errors. In other words, the odds against the residuals 
being different from zero by “pure chance” are all less than 20 to 1; there- 


fore, the conclusion that the true values are zero is acceptable. 
Now the nature of the factor pattern plan of the five variables which was 
conjectured earlier from an inspection of Table 6.4 has been verified by the 


more objective statistical procedures. In accordance with these findings, 
the new pattern plan may be formulated as in Table 6.7. 


may then be safely conclu 
from zero. The probabiliti 
greater than .05, the standar 


TABLE 6.7 


NEW PATTERN PLAN FOR 
FIVE VARIABLES 


Variable Fo Fi Di 
аю ац ац 
аю аш j| 
аз йа [oe 
ао Ош |е: зе 
"EE ds 


* Computed from the intercorrelations of all 17 variables; see ibid., Table 9. 
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The zoefficients ад (е = 1, 2, 3, 4) may now be computed by means of 
formula (6.16), which reduces to 


anc 1 Ж Тек 
М 3 Tik 


for the present case. They are as follows: 


:48-1,2,3,4, j <k, jke) 


an = V}(.32891 + .41300 + .39688) = V.37960 = .616, 
аз = V34(.58328 + .46452 + .56052) = V.53611 = .732, 
an = V3(.49624 + .41125 + .51639) = V.47463 = .689, 
аа = V3(.48164 + .50120 + .39915) = V.46066 = .679. 


It is evident that the triads involved in the calculation of any one of these 
coefficients are sufficiently constant so that the conditions (4.13) for one 
common factor are satisfied by the intercorrelations of the first four vari- 
ables in Table 6.4. The hypothesis of the single factor Ё, through these 
four variables is then quite adequate. 

There remains the calculation of the doublet coefficients in the plan of 
Table 6.7. If, as indicated earlier in this section, one standard deviation is 
allowed for chance error and the remainder of the residual correlation .274 
is divided equally between variables 1 and 5, then the communality of 
variable 1 exceeds unity, implying an imaginary unique factor. To circum- 
vent this difficulty, allow only one-tenth of the total variance of variable 1 
to the doublet Dı, making its coefficient di; = .316. This arbitrary pro- 
cedure is permissible as one of the infinite variety of ways in which the 
coefficients of a doublet can be assigned. Тһе other coefficient then becomes 


JA — J 
dy, = 270—209 _ ө. 


Тһе final factor pattern, for the given five variables, may now Бе writ- 
ten as follows: 
zı = .691F, + .616Ё, + .316D, 
Za = .591Fo + .732F, 
za = .581F, + .689F, 
24 = .598F, + .679F, 
25 = .674Ғо + .649D; 


ыс ы т жы ыл... —————————M——————— 
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where, as usual, the unique factors have been omitted and the variables 
have been designated by z's without double primes.* All the final residuals, 
with the factors Fo, Г, and the doublet D; removed, are insignificant so 
that no further factorization is required. The modification of the original 
bi-factor plan exhibited in the present example is, perhaps, one of most fre- 
quent occurrence. The analysis is simple and direct and was given in such 
detail here merely to bring out and clarify various procedures. In practice, 
the investigator would come to a conclusion as to the nature of the new pat- 
tern plan much more quickly: he might, at times, apply the B-coefficient 
technique; at other times, the tests of significance; while still more often, 
he might be able to formulate the new pattern plan simply by inspection of 


the general-factor residuals. 


6.10. Bi-factor Solution for Twenty-four Psychological Variables 


To round out the ideas of this chapter, a complete bi-factor solution for 
the data of 2.8 will be presented. The details of the straightforward analysis 
are given in outline form in Appendix C, while some of the refined tech- 
niques of modification and tests of significance will be discussed in this sec- 
tion. For purposes of reference, and to avoid repetition, the complete final 
factor pattern is given in Table 6.8. The manner in which the factor coeffi- 
cients were obtained will be explained in the sequel. 

In the outline of calculations of Appendix C the order of analysis is de- 
scribed in detail, beginning with the grouping of tests from the original 
matrix of correlations. This leads to step 17 in which the general-factor 
coefficients are calculated, and these are the values in column B, of Table 
6.8. The next stage of analysis involves the elimination of the general fac- 
tor, yielding residuals from which the group-factor coefficients are calcu- 
lated. The coefficients of Bi, Bs, and Вз are computed in steps 21 to 23. 
It is evident from Table C.6 that the general-factor residuals among the 
tests of G; are insignificantly different from zero and so may be considered 
as final residuals, obviating the original hypothesis of a group factor for these 
tests.] This leaves the group С. to be analyzed. 


* See n. *, р. 95. 

t With this modification of the original pattern plan, it is evident that there are addi- 
tional terms which can be included in formula (6.14) for the calculation of any one of the 
general-factor coefficients. These additional terms arise because the tests of Gs measure 
only the general factor and no group factor. Thus any two of the Tests 20 to 24, together 
with any other test of the battery, involve only one common factor. Since each general- 
factor coefficient is based on а large number of correlations, however, the addition ofa 
few more terms into the average of more than a hundred terms will not change its value 
much. For this reason it is not necessary to recalculate the general-factor coefficients. 
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From an inspection of the general-factor residuals among the tests of 
С. it would appear that Tests 14 to 19 do not measure a single common 
factor. Instead, it would seem that 17, 18, and 19 belong together; 14, 15, 
16, and 17 involve another factor; and the residuals for 14, 15, and 16 with 
18 and 19 raight be final residuals. These changes will now be justified. 


TABLE 6.8 
BI-FACTOR PATTERN FOR TWENTY-FOUR PSYCHOLOGICAL TESTS* 


General | Spatial Percep- | p Associ- 
Test? Deduc- | Rela- | Verbal | tual Tee, ative | Doublet | Unique 
tion tions Speed | 1110 | Memory 
j Bo В\ Bi Bs Bi Bs Di U; 
.589 ‚647 
.357 ‚889 
‚401 .781 
‚463 .828 
.582 .576 
.575 .597 
.534 .463 
.624 .686 
.560 .540 
.388 ‚599 
.521 .707 
.404 .652 
.576 .690 
.388 .743 
„851 .806 
.496 .793 
.422 .670 
.515 .718 
.442 .853 
.644 ‚765 
‚645 ‚764 
‚644 |.. ‚765 
.734 |.. .679 
.712 .596 
factor...... 6.874 | 0.645 | 1.678 | 1.185 | 0.779 | 0.539 | 0.275 |....... 


* This solution differs slightly, in the group factors, from the one given by Holzinger and Swineford, 
а Siudy in Factor Analysis, p. 33, since it was computed independently and the refined statistical ШО 
were employed here. 


> The names and brief descriptions of the tests appear in Appen. В.1. 


From step 24 in Appendix C it is evident that the general-factor residuals 
2mong the Tests 14 to 19 do not form a matrix of rank one. It will next be 
shown that the residuals for Tests 14, 15, 16 with 18 and 19 are insignifi- 
cantly different from zero. The largest of these six residuals is?isis = .153; 
if this can be shown to deviate from zero by chance errors of sampling, then 
the others would certainly be accepted ás such. To make this statistical 
test, the standard error of the residual is required. This can be obtained 
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from Table H.2 of Appendix H for N — 145 and p — .30* by linear inter- 
polation giving c; — .113. The ratio of the residual to its standard error is 


for which the probability P — .1770, obtained by linear interpolation from 
Table H.4. This result means that in sampling, assuming a true value of 
zero, the value .153 would be exceeded in absolute value in 17.70 per cent 
of the cases, or that the observed value is not significantly different from 
zero. Stated another way, the odds against the true value of the residual 
being different from zero by “риге chance" is only .8230 to .1770 or 4.65 
to 1, so that it is quite likely that the deviation from zero is due to random 
sampling. Since the largest of the six residuals is insignificantly different 
from zero, it may safely be assumed that all six are insignificant. Hence no 
linkage beyond the general factor is required between any one of the Tests 
14, 15, 16 with 18 and 19. 

The modification of the original hypothesis for the tests of С; now clearly 
points to a factor for Tests 14, 15, 16, and 17, and another factor for Tests 
17, 18, and 19. This conclusion is also corroborated by the B-coefficients 
for these two subsets of tests, namely, 


B(14, 15, 16, 17) — 202 and B(17,18, 19) 2 177, 
where the entire set of general-factor residuals for the six tests was em- 


ployed. The new pattern plan may then be written as in Table 6.9. 


TABLE 6.9 


NEW PATTERN PLAN FOR 
SIX VARIABLES 


Test Bo Bi Bs 
ац,о ац, 4 
415,0 015,4 
016,0 16, 4 
0,0 ап, 
Ou Jes ens 
бәл ns ae ew wo 


* The average correlation is obtained from the entire set of raw correlations of Table 
2.2 by summing all the column sums and dividing by twice the number of different corre- 
lations. In symbols, this is 


papali 
e = 30 


Bem jk=1,2,..., 24, jk) = 159299 = зур, 
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According to the revised plan, the general-factor residuals for Tests 14 
to 19 are assumed to form a matrix of rank two. Now the direct method of 
analysis, as exhibited in 4.7, may be employed for this case; or else appro- 
priate tests may be selected, from the total set of six, which taken alone 
have a correlation matrix of rank one, and then the method of triads may 
be applied. The solution by the method of triads is given in Appendix C 
and the coefficients recorded in columns B, and B; of Table 6.8. For com- 
parison, the solution has also been obtained by applying formula (4.22) and 
the procedure of 4.7. The coefficients obtained by the latter method are 
listed in Table 6.10 under Bj and В; to distinguish them from the values 
obtained by the former method. A comparison of the two sets of cocfficients 


TABLE 6.10 
COMPARISON OF TWO SETS OF В, AND В, COEFFICIENTS 


is also given in this table. The differences are obviously insignificant, and, 
because the values are obtained more easily by the method of triads, these 
are used in the final pattern. 

The original correlations are now accounted for by means of the general 
factor and five group factors except for опе“ large residual, 5,954 = .255. 
To test this residual for significance, its standard error is required and will 
be computed by means of formula (6.36) and compared with the value from 
Table H.2. A similar comparison was made in Table 6.6, but in using for- 
mula (6.36) the standard errors of the coefficients were obtained from 
Table H.1. Now formula (6.34) will be used to get the standard errors of 
the factor coefficients, thus illustrating the computation by means of this 
formula. 


* There is one other large general-factor residual, namely, 7, = —.231. But there 
would be no point in applying statistical tests of significance to this value, for the facts 
of the problem clearly indicate that this value must be insignificant. The original corre- 
lation was only rs, о = —.075, and it is due to replacing this value by zero in the calcula- 
tion of the general-factor coefficients that the residual for these tests is increased nega- 
tively. 
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First the standard error of азоо will be calculated. It will be recalled 
that there are л = 24 variables and that there are 


р= 4 р=5 P4 р= 6, ps=5 


variables in the respective groups G, to Съ. Now 10667; so that the system 
((10, j, k); jeG,, keG,, s < t, 5,1 = 1,2, 4, 5) may be used in evaluating азо, o. 
The number of triads which can be set down frcm such triples is easily 
found to be 
v = pi(pa + pa + рь) + ps(pa + ps) + p4(ps) , 
= 4(16) + 5(11) + 6(5) = 149. 


The component parts of triads formed from such sets, designated by a — 
тор b = ти, € = тук according to (6.19), must be averaged over the 
v values. It is necessary to write out neither the individual values of a, b, 
and c, as was done in Table 6.2, nor even the indices and triads for the pres- 
ent example because the coefficients have already been computed in Ap- 
pendix C and the necessary sums of correlations can be picked out of Tables 


C.1 and C.2. Thus, 


a= i [(pa + pat рв) (710,01) + (Pa + p3)Z (9,553682) + Ds= (rio, 55 3€G4)] 
v 


14 
_ 23.947 _ 
= = --1607, 


where the appropriate sums of correlations for Test 10 with the tests of 
groups G1, Ga, and G are obtained from row 10 of Table C.1. Similarly, 


В = 1 хаа Кеб) + (Pit ра) (rio, Re) + Prt pi PIE (пов; 100) 
v 


=l [4(1.215) + 9(1.246) + 15(1.531)] 


_ 39.039 _ 
149 | 
The correlations с = тук are precisely those involved іп the denominator of 


formula (6.14) for the evaluation of a factor coefficient. Hence, employing 
the denominator for 1073 from Table C.2, 


.2620 . 
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Now, substituting the values of à, b, and с into formula (6.34), the standard 
error of the coefficient aio,o becomes 


(2 + 2.15821 + 5.73668 — .63945 — 1.69974 + 1.83415 
— .74073 + .08779) 


2 
баю, = из 


. 8.73691 


z0 = 01506, 


and 
бав = 193. 


Before the standard error of the residual 719,24 can be calculated by for- 
mula (6.36), the standard error of аза, о must also be obtained. Proceed- 
ing as before, there are 


pi(pa + Рз + ps) + px(ps + ps) + ps(ps) 
= 4(15) + 5(10) + 4(6) = 134 


v 
sets from which triads may be formed in evaluating the general-factor coeffi- 
cient for 24«G,. The average of the correlations a = 74,; is given by 


а= 1 (ра + pa + рта) JeGr) + (ра + ра) (та, 53 3682) 
+ раХ(тза,;; JeGs)] = 3393 , 


where the sums are obtained from row 24 of Table C.1. In like manner, the 
average of the correlations b = 724, is given by 


b= 1 [p2 (ra, ki кеб) + (pi + рг) (74,05 кеСз) 
+ (р, + р + p3)E(rass; keG4)] = .3561. 


From Table С.2, the denominator for any test of Gs is 31.455, so that 
t= 2347. 


The square of the standard error of the coefficient аза, then becomes 


CM = 
2 + 4.05973 + 4.47171 — 4.17994 — 4.60413 + 9.34599 — 2.57403 + 1.06010 
4(145) 
= .01652, 
and 


ба = 129. 


Мир a OA. салм с. тыл e ee t " 
/——— "- 


— 
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Now the preceding values for the squares of the standard errors of the 
factor coefficients may be substituted into formula (6.36), giving 


iat 2)2 
ыт = + .15036(.01652) + .50705(.01506) , 


T10,24 1 


where the first term is the square of the standard error of 710,24 = .531 as 
given by (6.21). The standard error of the required residual is then 


аы, = V.01368 = .117. 


Previously, on page 145, the value for the standard error of any general- 
factor residual, according to Table H.2, was shown to be о = .113. There 
appears to be a remarkably close agreement between the value obtained by 
means of formula (0.36) and that from Table H.2, just as in the case of the 
example of the last section. 

The significance of the residual 70,24 = .255 сап now be tested, knowing 
that its standard error is .117. As a matter of fact, these figures formed the 
basis of the illustrations in 6.8, where it was shown that the residual .255 
was significantly different from zero. Since all other residuals, with the gen- 
eral and the group factors removed, for Tests 10 and 24 are insignificant, а 
doublet may be postulated between these tests. Again, in 6.9, the doublet 
weights were shown to be 


dioi = dai = .371. 


This completes the analysis into common factors. 

It may be noted that there are no insignificant values in Table 6.8. A 
rough estimate of the standard error for any one of the coefficients may be 
taken from Table H.1 for № = 145 and р = .30, namely, ca = .109. The 
smallest coefficient is .278, which is 2.55 times this standard error, while 
any one of the other coefficients is more than three times its standard error. 
Hence every coefficient in the final factor pattern is definitely significant. 

The unique-factor coefficients can readily be obtained by applying the 


formula 
a; = V1 — №, 


where h? is the communality of test z;. The communality and uniqueness of 
each test are given in Table 6.11, while the square root of the uniqueness is 
entered in Table 6.8 as the coefficient of its unique factor. Table 6.11 also 
contains the unreliability and the specificity so that the apportionment of 
the unit variance of each test can be seen at a glance. In addition, the index 
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of completeness of factorization is also given for each test. This shows the 
percentage of the reliability variance accounted for by the common factors. 

The index H; may be used as one standard for the adequacy of a factorial 
solution. The analysis of psychological tests into common factors should 
not be carried to the point where real specific factors disappear. In the pres- 


TABLE 6.11 
APPORTIONMENT OF TEST VARIANCES 
Commu- Relia- " Unrelia- ТРАЕ? Index of Ғас- 
Test nality bility Uniqueness bility Specificity тесеру үйе) 
j м M а=1-№ | de1-rj | Біза)-«) |Hj-100— 
tjs 
‚581 .756 ‚419 .244 .175 76.9 
.209 .568 .791 .432 .359 36.7 
.890 .544 .610 .456 .154 71.7 
.315 .922 .685 .078 .607 34.1 
.668 .808 .932 .192 .140 82.7 
.643 .651 .357 .349 .008 98.8 
.786 .754 ‚214 ‚246 —.032 104.2 
.530 .680 .470 .320 .150 71-9 
.708 .870 .292 .130 .162 81.4 
.503^ .952 .497 ‚048 ‚449 52.8 
“500 1712 500 288 212 70.2 
.575 .937 .425 .063 .362 61.4 
.524 ‚889 .476 m .365 58.9 
.448 .648 .552 .352 .200 69.1 
.350 .507 .650 .493 .157 69.0 
.871 .600 .629 .400 .229 61.8 
.551 .725 .449 .275 .174 76.0 
.484 .610 ‚516 390 126 79.4 
.273 .569 .727 481 .296 47.9 
.415 .649 ‚585 851 ‚234 63.9 
416 ‚784 ‚584 ‚216 ‚368 58.1 
‚415 .787 .585 .213 .872 52.7 
.539 .931 .461 .069 | .892 57.9 
‚5075 .836 .493 .164 .829 60.6 


«Тһе communality with the doublet Di included is .641. 
b The communality with the doublet Di included is .615. 


ent example, there is one value of H; exceeding 100 per cent, and which is 
probably due to chance. The preceding checks, and the one to follow, indi- 
cate that the factorization has not been carried too far, so that this dis- 
erepancy may be attributed to chance errors in the reliability coefficient. 
Only when several values are greater than 100 per cent, for high reliability 
coefficients, should the factorization be changed.* 


* An example where even this crude test would indicate that the factorization has been 
carried too far is available in Thurstone, Primary Mental Abilities, where the commu- 
nalities for seven tests appreciably exceed the reliabilities. 
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Аз a final check on the adequacy of the factor pattern, the criterion of 
2.5 will be applied. This requires the standard deviation of the series of 
final residuals to be less than, or equal to, the standard error of a zero corre- 
lation for a sample of N = 145. The frequency distribution of the final 
residuals (of Table C.6) is presented in Table C.12. The mean and standard 
deviation for the residuals are also given in this table. Now the standard 
error of a zero correlation is 


Hoe .0830 . 


ome 145 


Evidently the required inequality is satisfied, and the factor pattern may 
be regarded as acceptable. 

Various standards for adequacy of factorization, or "when to stop fac- 
toring,” should yield the same result for a particular problem if they are to 
be equally valid. Of course, if one of the criteria is more crude than the 
others, less emphasis should be placed on the changes which it may indi- 
cate. For the present example this would mean that, if the difference be- 
tween the standard deviation of the final residuals and that of a zero corre- 
lation were considered too large,* the pattern would still not be revised be- 
cause the more exact tests by means of the sampling error formulas indicate 
that the factor pattern is a valid description of the original variables. Asa 
matter of fact, the difference .0175 = .0830 — .0655 usually would be con- 
sidered insignificant, so that this check also indicates that the factorization 
has been carried to the proper stage. The important thing to note is that 
one cannot merely “turn a crank” and come out with a valid solution, al- 
though according to some crude, cover-all standard it may be deemed ac- 
ceptable. The experimenter should use all the theoretical and factual knowl- 
edge at his disposal at each stage of the analysis to produce the best solution 
from the given techniques and data. 

Before leaving the present example, a word about the naming of factors 
may be in order. It will be recalled that the fundamental purpose of factor 
analysis is to comprehend a large class of phenomena (the values of a set of 
variables) in terms of a small number of concepts (the factors); and for the 
present time, at least, this description is taken to be a linear function of the 
factors. In a mathematical or physical theory it may be sufficient to know 
that twenty-four variables can be described linearly іп terms of only six new 
hypothetical ones—that is usually quite an accomplishment, and it is of 
little concern as to what the six new variables are called. But in the biologi- 
cal and social sciences—psychology, for example—it is usually demanded 


* The standard is only a rough one, and the permissible difference is not known. 
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that these new variables be named in order that the solution can be given а 
more “practical interpretation." 

Тһе coefficients of a factor pattern indicate the correlations of the vari- 
ables with the respective factors and furnish the basis for naming them. In 
the case of oblique factors, to be discussed in later chapters, the structure 
furnishes the correlations of the variables with the factors, and so it is simi- 
larly employed in naming the factors. The investigator is guided by the 
magnitude of the factor weights in the selection of appropriate names for 
the factors. The name selected is usually suggested by the nature of the 
variables having the largest correlations with the factor under considera- 
tion. This name should be consistent with the nature of the remaining vari- 
ables which have a low correlation with the factor. 

Тһе common factors for the example are named from the pattern given 
in Table 6.8 and the brief descriptions of the tests in Appendix B.1. The 
factor Во has positive weights throughout and correlates highest with such 
deductive tests as Series Completion (23), Woody-McCall Arithmetic (24), 
Problem Reasoning (22), and Word Classification (8). Hence Во might be 
called a “general deductive factor." This name is consistent with the nature 
of the remaining variables—those involving a lesser amount of deductive 
ability have correspondingly smaller factor weights. 

Тһе remaining common factors are named from the subgroups of tests 
which have significant correlations with them. The first group factor is 
named from the "spatial" subgroup (Tests 1-4), the second from the “уег- 
bal” subgroup of tests, and similarly for the remaining factors. The names 
of the six common factors are indicated in Table 6.8. In addition to the 
common factors, there is one unique factor for each of the twenty-four tests. 
If a name were desired for any unique factor, it would be obtained from the 
description of the particular test. The only unnamed factor is the doublet 
D; involved in Speed of Adding (10) and Woody-McCall Arithmetic (24). 
This doublet appears to measure “arithmetical speed," which might appear 
as а more significant factor if more tests of this type were introduced in а 
battery to experiment for this purpose. 

For future work with this factor pattern the doublet will be dropped from 
consideration, since, as was remarked before, it takes at least three variables 
to define a factor. The six common factors may be referred to by means of 
symbols or the descriptive names, which are tentatively assigned for that 
purpose. The partieular name by which a factor is designated, however, 
should not raise an issue for dispute. If another investigator chooses to call 
these factors by other names, he is free to do so. Тһе naming of factors is 
not a problem of factor analysis, which is a branch of statistics, but some 
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descriptive names may be highly desirable in a particular subject matter for 
purposes of classification. 


6.11. А Bi-factor Solution for a Reduced Set of Variables 


It should be evident that any factor solution is dependent upon the 
sample of individuals and variables. When either of these is altered, it is to 
be expected that the solution will not remain strietly invariant but may 
exhibit a certain amount of stability. This stability will be illustrated for 


TABLE 6.12 
BI-FACTOR PATTERN FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
Common FACTORS 
T Соммо- 
ЖӘ» General Spatial Percep- NALITY 
Deduc- . Rela- Verbal tual 
tion tions Speed 
2 Bo Bi B: Bi Aj 
.558 
.203 
.962 
.914 
.646 
.641 
.750 
.571 
.758 
.554 
.449 
-531 
.599 
Contribution 
offactor..| 3.538 | 0.596 | 1.441 | 1.360 |........ 


the case of a fixed sample of individuals and a reduction in the number of 
variables. 

A bi-factor solution for the first thirteen of the preceding set of twenty- 
four psychological tests has been obtained by the methods of this chapter 
and is presented in Table 6.12. The variables of this subset were selected 
so that the solution should contain three group factors and include only 
about half of the original variables. 

As might be expected, the form of solution given in Table 6.12 agrees 
with the corresponding portion of that given in Table 6.8. The respective 
Coefficients for these two patterns are in close agreement throughout. The 


“а 
. 
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largest discrepancy occurs in the case of the general-factor coefficients for 
variable 10, which have the values .388 and .244, respectively. Inasmuch 
as the standard error of азо, о = .388 has already been shown to be .123, the 
difference .144 is regarded as insignificant. The communalities for corre- 
sponding variables are also in close agreement, the greatest discrepancy 
being .075 for Test 13. 

The foregoing illustration is, of course, an example of the stability of one 
type of solution, when the set of variables is reduced. This type of stability 
will also be illustrated for other forms of solution for the same data in Chap- 
ters X and XI. 


CHAPTER VII 
THE PRINCIPAL-FACTOR SOLUTION 


7.1. Introduction 

The principal-factor method was developed by Hotelling* at the sug- 
gestion of Kelley, who since has developed an alternative procedure,] based 
on covariances instead of correlations. In a still more recent paper Ho- 
tellingt has presented an improvement on his original scheme for calcula- 
tion of principal factors. Hence methods available for calculation include 
Hotelling's first procedure, Kelley's improvement by use of а simple mathe- 
matical device, and Hotelling’s latest simplified scheme. For simplicity 
this latter form of computation only is treated here. 

It will be noted that the term “factor” is employed here, as elsewhere in 
the text, rather than “component,” which is used by Hotelling and Kelley. 
This seems advisable especially in later chapters where comparisons of dif- 
ferent solutions are made. Hotelling's objection to the word “factor” arose 
from its possible confusion with the mathematical term, but general usage 
would appear to favor retention of "factor." 

Аз indicated in 2.5 the communalities of the variables are usually the 
desired portions of the variances to be analyzed. In 7.2, therefore, various 
methods for estimating communality are presented. The problem of ob- 
taining these estimates is the same for both the principal-factor and the 
centroid solutions, so the discussion in this section applies also to the meth- 
od treated in Chapter VIII. 

The essential portions of Hotelling's theoretical treatment of the princi- 
pal-factor solution are given in 7.8, and certain proofs of the fundamental 
development are presented in Appendix B.9. The method of analysis is 
applied to a matrix of correlations with communalities in the diagonal rather 
than ones or reliabilities as Hotelling suggests. Although he has furnished а 
treatment of the sampling problem, it is not applicable in the present de- 
velopment which involves communalities. When appropriate estimates of 
the communalities are employed, it is shown in 7.2 that the problem of 
"when to stop factoring" is obviated. 

* Harold Hotelling, "Analysis of & Complex of Statistical Variables into Principal 
Components," Journal of Educational Psychology, 1933, pp. 417-41, 498—520. 

1 Truman L. Kelley, Essential Traits of Mental Life, 1935. 

1 "Simplified Calculation of Principal Components," Psychometrika, 1936. 
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Except for the choice of communalities, the solution of a principal-factor 
pattern can be reduced to & routine procedure. Àn iterative scheme for 
calculation is described in 7.4, and a detailed series of steps for computation 
is given in Appendix D. In illustrating these steps a set of only eight vari- 
ables is employed inasmuch as this is a sufficient number to clarify the pro- 
cedure. Furthermore, it is usually advisable to restrict the application of 
the direct principal-factor solution to small sets of variables because of the 
excessive amount of calculation when the number is large. In Chapter IX 
a method will be given for obtaining a principal-factor solution by rotation 
from another solution. If the latter method is employed, the labor is greatly 
reduced, and the principal-factor form of solution may thus be made feasible 
for larger sets of variables. 

In section 7.5 the solutions for four sets of variables are presented. The 
first of these analyzes the eight physical variables used for illustration in 
Appendix D. The next application is made for the twenty-four psychologi- 
cal tests of 2.8, in spite of the large amount of calculation entailed. It was 
originally intended to employ this example in the outline of steps in Appen- 
dix D, but because of the bulk of tabular material it was feared that the 
reader would fail to see the trees because of the forest. The detailed com- 
putations of this example are, therefore, not included in the text, but only а 
portion of the pattern is exhibited in 7.5. The third illustration is given 
for eight emotional traits studied by Cyril Burt. The final example is taken 
from the field of political science. А discussion of the factors obtained іп 
each example is presented to indicate the effectiveness of the principal- 
factor form of solution. 


7.2. Estimation of Communalities 


It was pointed out in 2.5 that the portions of the variances to be factored 
are determined by the diagonal elements of the correlation matrix. When 
ones are put in the principal diagonal of this matrix, the resulting descrip- 
tions of the n variables are in terms of n (sometimes fewer) common factors. 
This was the approach of Hotelling in his method of analysis. In the present 
treatment, however, communalities are the basic quantities to be analyzed 
in accordance with the assumed composition of variables (2.4). The de- 
velopment of the principal-factor method in the following section is based 
upon the latter assumption of the composition of the variables, and hence 
good estimates of the communalities are required. 

Appropriate communalities, and hence the entire factor pattern, can be 
obtained directly by the methods of Chapter I'V if the rank of the correlation 
matrixissuitably approximated. Thusin the bi-factor method the rank is ap- 
proximated by means of the grouping of variables. Then the pattern plan is 


THE PRINCIPAL-FACTOR SOLUTION 157 


postulated, and the coefficients are computed directly by formula. If, in- 
stead of approximating the rank, estimates of the communalities are em- 
ployed, it is possible to obtain a solution with a number of factors dependent 
upon such estimates. In the factorial analyses of this and the succeeding 
chapter, the procedure of estimating communalities is generally employed. 

When no such estimates are feasible, owing to the nature and lack of de- 
sign in the set of variables, an arbitrary method will be presented in 8.4 in 
which no explicit use of the diagonal elements is made, and the adequacy of 
the resulting pattern is tested from the final residuals. 

One method for estimating the communality of a variable z; is to select 
the highest correlation among all its correlations with all the other vari- 
ables of the given set. Another method is to employ as an approximation to 
the communality a triad А = rj;rji/rii, where туь and rj; are the two highest 
correlations for z;. Still another estimate is given by the average of all the 
correlations of each variable. These "arbitrary estimates" of the commu- 
nalities have been employed by workers in multiple-factor analysis, who 
contend that the first methdd is sufficiently accurate when applied to sets 
of twenty, or more, variables. 

A still more satisfactory procedure for estimating communality can be 
obtained by employing the methods of Chapter IV. By means of the group- 
ing of variables, the approximate rank of the correlation matrix may be 
assumed. In contrast to the bi-factor procedure, however, no pattern plan 
is postulated. The groupings of variables are obtained merely to approxi- 
mate the rank in order to get suitable estimates of the communality. For 
any rank, thus determined, the direct method of obtaining the commu- 
nalities, as described in 4.6, is theoretically most desirable. This method 
involves the calculation of the average of all possible expressions (4.27) for 
each communality, when m is the rank of the correlation matrix. For small 
sets of variables it is expected that there will be only a few factors.* It may 
then be sufficiently accurate to assume rank one or two and employ the 
method of Chapter IV for calculating approximations to the communalities. 
In practice, when the rank exceeds two, the method of Chapter IV is not 
feasible because of the complexity of the formulas and large amount of com- 
putation involved. One simplification of this technique is accomplished by 
considering the section of the correlation matrix corresponding to a sub- 
group of variables as approximately of rank one. The estimates of the com- 
munalities may then be readily calculated by means of formula (6.9) ap- 
plied to the subsets of variables. This procedure seems much more satis- 
factory than the estimation of the communality by a single triad involving 
the two highest correlations of a given variable. 


* See 4.2. 
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In order that a subset of variables shall produce a correlation matrix of 
rank one, it is necessary that the vectors representing these variables lie in 
a space of one dimension. Since such a configuration of vectors is not ordi- 
narily to be expected for a complete set of variables, the rank usually will 
be greater than one. Ап even better estimate of the communality than the 
preceding one may be obtained by selecting appropriate variables out of the 
total set so as to yield a matrix of rank one, and applying the bi-factor tech- 
nique described in 6.4. Although the bi-factor pattern is explicitly obtained, 
it is employed here only as a means of obtaining good estimates of the com- 
munalities to be used in the calculation of other types of factor solutions. In 
general, then, the communalities from any satisfactory solution may be 
taken as good estimates in further factorial analyses. To contrast an arbi- 
{тагу estimate for а variable involving only a few correlations with that 
based upon the entire correlation matrix, the latter will be designated as а 
“complete estimate" of the communality. 

Inasmuch as the arbitrary estimates may not be close to the desired com- 
munalities, it has been suggested* that the diagonal values be changed in 
each table of residual correlations for the calculation of successive factor 
coefficients. For example, when the highest correlation for each variable 
is taken as its communality, the perplexing problem as to ‘‘when to stop 
factoring” arises. In the matrix of residual correlations with the first factor 
removed, the highest value in each column would again be used for the 
diagonal entry (in place of the value actually computed). By continuing 
this process, a large number of factors (relative to the number of variables) 
may be obtained, with some significant coefficients even for the last of many 
factors. 

When the complete estimates of the communalities are employed, the 
foregoing difficulty disappears. Since these estimates are the best available, 
the diagonal values are not altered as in the case of arbitrary estimates. 
Then, in a practical sense, the question of “when to stop factoring” no 
longer arises. The factoring is carried to the stage where nearly 100 per cent 
of the total estimated communality is analyzed. When this is accomplished, 
the number of common factors is relatively small; and, if additional factors 
are obtained, practically all coefficients are insignificant. An illustration of 
this “convergence” to a small number of common factors is given in 8.3, 
where a centroid pattern for the twenty-four psychological tests is obtained. 

The most important advantage in employing the complete estimates of 
the communalities in the calculation of a principal or centroid factor pattern 
arises in connection with the statistical adequacy of these solutions. Such 
estimates are based upon the bi-factor solution, for which sampling error 


* L. L. Thurstone, The Vectors of Mind, 1935, p. 113. 
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formulas are given in Chapter VI. If the bi-factor pattern is justified by the 
various standards, including the sampling formulas, the resulting commu- 
nalities are also validated. Then, by employing these communalities in any 
other form of analysis, further tests of adequacy of solution are obviated. 

From the geometrical point of view, the fundamental problem of factor 
analysis is the determination of the common-factor space. Once this space 
has been determined, any reference system (corresponding to a set of fac- 
tors) can be transformed into another preferred type by means of а suitable 
rotation. In obtaining a bi-factor solution, the common-factor space is 
thereby fixed. Then in further analyses, employing the bi-factor commu- 
nalities, the common-factor space is predetermined. Any factor solution in 
this common-factor space may then be regarded as a linear transformation 
of any other. In actual practice, however, such alternative solutions are 
generally obtained by different statistical procedures. Illustrations of these 
geometric properties are given in Chapter XIII. 


7.3. Principal-Factor Method 

As indicated іп 5.2, when the point representation of a set of variables is 
employed, the loci of uniform frequency density are essentially concentric, 
similar, and similarly situated ellipsoids. The axes of these ellipsoids corre- 
spond to the factors in the principal-factor solution.* From an algebraic 
point of view, the selection of these axes is equivalent to choosing a set of 
factors in decreasing order of their contribution to the total communality. 
The analysis is begun with a factor F, whose contribution to the commu- 
nalities of the variables has as great a total as possible. Then the first- 
factor residual correlations are obtained, including the residual commu- 
nalities. A second factor Fs, independent of F, with a maximum contribu- 
tion to the residual communality is next found. This process is continued 
until the total communality is analyzed. 

A brief theoretical description of the principal-factor method will now be 
given, and this will be followed by a discussion of a more expedient form of 
solution in the next section. If the composition of a statistical variable} is 
again taken to be 
(7.1) 2; = apFı + aF: +... + аҙ” m 

G= 1,2,..- 7)» 


with the unique factor omitted, the communality of z; is then given by 
(7.2) № = а, + ah +... +a +... 0m. 


* Hotelling, “Analysis of a Complex of Statistical Variables into Principal Com- 
ponents,” Sec. 3. 
1 See n. *, р. 95. 


160 FACTOR ANALYSIS 


In general, the term aj, indicates the contribution of the factor F, to the 
communality of z;. The sum of the contributions of the first factor F to 
the communalities of the т variables is 


(7.3) A, = а, Ғаһ-ғ...-ғаз. 


The object of the present method is to choose the coefficients ал so as to 
make A; a maximum, subject to the restrictions that the correlations are 
reproduced by the pattern (7.1). The conditions may be expressed as fol- 
lows: 


(7.4) Tj = Tjk = D анам (,%-1,2,...,т). 
t=1 


In the succeeding analysis the reproduced correlations rj, will be replaced 
by the corresponding observed correlations т;к. As indicated in Chapter VI, 
this procedure implies the assumption of zero residuals. 

By the methods of the calculus it is possible to maximize A; under the 
given conditions.* The resulting system of equations for the solution of the 
unknowns aj may be written as follows: 


(hi — X)an + rian + тва -... + Tindn = 0, 
radu + (А2 = \)аз1 + T203 +... + Tanan = 0, 
(7.5) Taan + Taaa + (hj — Man +... + Tanan = 0, 


Tidi + тла + тма +... + (AR— Man = 0, 
where A is a parameter independent of the a’s. A necessary and sufficient 
condition for the system of equations (7.5) to have a solution (in which not 


all unknowns are zero) is the vanishing of the determinant of the coeffi- 
cients} of the ал. This condition may be written in the form 


(8-02) та Тіз mane Wig 
Tn (8-0) т» T Жой 

(7.6) Та Таз (h— №) ... Tan =0. 
та Та? Таз e. (8-Х 


* For & proof see Appen. B.9. 
1 L. E. Dickson, Modern Algebraic T'heories, 1930, p. 61. 
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An equation of the form (7.6) is known as a characteristic equation, some of 
its properties being that all the roots are real and that a g-fold multiple root 
substituted for А in (7.6) reduces the rank of the determinant to (n — 0). 

When a simple root of the characteristic equation is substituted for \ in 
(7.5), а set of homogeneous linear equations of rank (n — 1) is obtained. 
This set of equations has a family of solutions, all of which are proportional 
to one particular solution. It follows* from the analysis for maximizing Аі 


n 
that the factor of proportionality is № = Ma = Ai. Hence А}, which is 


j-1 
to be maximized, is equal to one of the roots of the characteristic equation, 
namely, the largest root №. 

The problem of finding the coefficients ад of the first factor Ё,, which 
will account for as much of the total communality as possible, is then solved. 
The largest root X; of (7.6) is substituted into (7.5), and any solution an, 
аз, ... , ал ls obtained. Then, to satisfy the relation (7.3), these values are 
divided by the square root of the sum of their squares and then multiplied 
by V^. The resulting quantities are 


RAM 


ат 12 1,2 зар ЫН 
HC We Ful Т... ah s | 


(7.7) 


which аге the desired coefficients of F; іп the factor pattern (7.1). It may 
be observed that these values of aj: satisfy the condition (7.3); for, upon 
squaring the expressions (7.7) and summing, there results 


= M (ali + а, +... + 021) 
2 — Alen 21 m) е 
жаа тан ата "ады 


When the largest root of the characteristic equation is а g-fold multiple 
root (i.e., the q largest roots are equal to X), a set of n homogeneous linear 
equations of rank (n — q) is obtained upon substituting this value for А in 
(7.5). This system of equations has 4 linearly independent solutions, while 
every other solution is linearly dependent on them. Designating these solu- 
tions by 


ап, 4%, «++, аһ, 
012, 492, «++, аһ, 
ац, 4а, ..., Qn, 


* See Appen. B.9, eq. (B.28). 
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they may be selected so that they are "orthogonal" to each other, in the 
sense that 


Хе =a 
ігі Bb = 1,2) 6425 

(7.8) { ( ed. »I 
Maas =0 


j-1 


Тһе 4 sets of solutions may then be taken as the coefficients of q independ- 
ent factors Fi, Ғә,..., Ё, which account (equally) for as large as possible 
a part of the total communality. Inasmuch as this case is very unlikely to 
arise in actual practice, it will not be treated explicitly in the subsequent 
analysis. 

Having determined the coefficients aj of the first factor Fi, the next prob- 
lem is to find a factor which will account for a maximum of the residual 
communality. In order to do this, it is necessary to obtain the first-factor 
residual correlations. Furthermore, in obtaining still other factors the resid- 
ual correlations with two, three, ..., (m — 1) factors removed аге em- 
ployed, and hence a suitable notation is required. In the last chapter only 
residuals with the general factor removed (7) and final residuals (ғ) were 
explicitly indicated. A convenient notation for the residual correlation of 
тук with p factors removed is угу. Thus, when the first factor has been ob- 
tained, the first-factor residuals are written in the form 


(7.9) Wie = Tj — алам = азак + jas +... + айт. 


In determining the coefficients of the second factor Рз, it is necessary to 
maximize the quantity 


(7.10) A, = а, + а, +... + 4%, 


which is the sum of the contributions of F; to the residual communality. 
This maximization is subject to the conditions (7.9), which is analogous to 
the restrictions (7.4) in the case of the first factor. The analysis for obtain- 
ing the coefficients aj; is parallel to that for determining the first-factor 
coefficients, the only difference being that the residuals ,r;, are used in place 
of the correlations ту. Proceeding in this way, the coefficients of /1,/2, 
..., Fm are determined in the order of their contributions to the total 
communality. 

An important mathematical property of the principal-factor pattern is 
that of orthogonality. It has been indicated already that in case the largest 
root of the characteristic equation is a multiple root, this orthogonality is 


} 
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given by (7.8). The property of orthogonality holds, in general, for the 
coefficients of all the principal factors. These relations may be written as 


follows: 


Уа = № 
71 зі [nr ul 
мау п ssl Ў 
У) аһа, = 0 
ізі 


The properties (7.11) are useful in checking the numerical calculations of 
the factor coefficients. 

The foregoing description of the principal-factor method may be con- 
sidered as its logical basis but not as a scheme for actual computation. The 
direct solution of the characteristic equation and sets of linear homogeneous 
equations such as (7.5) would involve great algebraic difficulties. For this 
reason, an iterative method which requires only routine computations will 
be described in the next section. 


7.4. Iterative Form of Solution 

A simplified method for calculating a principal-factor pattern will now 
be developed. This treatment is based upon the two fundamental papers of 
Hotelling in which he first presents the iterative scheme and later gives an 
improved variation of it. Detailed steps for the numerical calculation of the 
pattern are given in Appendix D. 

The iterative form of solution yields a root of the characteristic equation 
and the corresponding coefficients in the factor pattern simultaneously. 
The roots appear in descending order of magnitude upon successive applica- 
tions of the iterative scheme. When complete estimates of the commu- 
nalities are employed, the number of roots required to account for this total 
communality is relatively small. The contribution of any factor, say, after 
95 per cent of the communality has been analyzed, would be expected to be 
insignificant. The iterative procedure, based upon complete estimates of 
the communalities, then affords an adequate factorization. 

In the iterative process an arbitrary set of » numbers is selected, and, 
after manipulating these numbers with the matrix of correlations, they are 
finally reduced to the desired coefficients of the first principal factor. Thus, 


take the numbers ап, азу...) anı аз the arbitrary set and consider the 

following transformation 

(7.12) aj, = S rnan (== 1,9)... 00) 
Еті 
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to the new set (ай, o4, ... , од). If the numbers ал are proportional to 
the direction cosines of any line through the origin, then the numbers a;, are 
proportional to the direction cosines of a new line (through the origin) cor- 
responding to the original line under the rotation (7.12). 

In general, the line associated with the numbers oj, is distinct from the 
line corresponding to the aj. There may, however, be lines which remain 
fixed under the transformation (7.12). Such invariant lines are those for 
which a quantity А exists such that 


(7.13) oj, = Ma 6 -212,...,9). 


Upon substituting the values (7.13) in (7.12), that expression reduces to 


n 
Aag = > трап = Таап + Tan t+... + Aja +... Tinan 
іі 


or 


(7.44) raan + ryan +... + (h — an t+... b rias = 0. 


As j takes on the values 1 to n in (7.14), it is readily seen that these equa- 
tions are identical with (7.5). Thus, for any invariant line, the direction 
cosines are proportional to a solution of (7.5), where X is a root of the charac- 
teristic equation (7.6). Hence it follows that the invariant lines are the de- 
sired principal axes. It is thus apparent that, if a set of numbers an, an, 

. , anı can be found which when substituted into the right-hand members 
of (7.12) produce relations (7.13), the numbers а, ад,..., ащ аге pro- 
portional to the direction cosines of the principal axes. The coefficients of 
one of the principal factors can then be obtained from the latter set of num- 
bers. Furthermore, № in (7.13) is the sum of the contributions of this factor 
to the communalities of the variables. 

In practice, of course, it cannot be expected that the arbitrary numbers 
aj will be so selected as to be proportional to the direction cosines of one 
of the principal axes. The iterative process then involves the use of the 
derived numbers aj, as a new set of arbitrary numbers in place of ал. Now 
if the numbers ej, are proportional to the direction cosines of any line 
through the origin, and if they are substituted in the equations* 


n 

(7.15) af = > жыш, (г -1,2,...,т), 
ізі 

* The symbol ası is employed instead of a;i corresponding to the notation in (7.12) 


because it will be found convenient for subsequent algebraic manipulation to retain j 88 
a subscript in the right-hand member of (7.15). 


in лаа а < — — 


—-— 
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then the quantities on the left will be proportional to the direction cosines 
of a new line (through the origin) corresponding to the original one under 
the transformation (7.15). This process is continued until the ratios among 
the quantities obtained at any stage converge to the corresponding ratios 
among the coefficients of F, to any specified degree of accuracy. The proof 
of the convergence of these ratios to those of the coefficients ад of the first 
principal factor is given by Hotelling.* А convenient procedure is to divide 
each of the trial values by a fixed one of them, say the largest. Then the 
next value obtained, corresponding to this number, will be an approxima- 
tion to the characteristic root Xi. 

Instead of calculating the successive values a/,, ад, etc., and substituting 
them in equations like (7.12) and (7.15), a modification will next be intro- 
duced which greatly accelerates convergence.| This simplification is ac- 
complished by the formal substitution of the a's in these equations. Thus, 
upon substituting the values for aj, from (7.12) into the right-hand member 
of (7.15), the latter equation takes the form 


n n 
Nem 
Qu = > Торан. 


k=1 j=l 


This expression may be written more simply as follows: 


(7.16) aff = > enon @ =1,2,...,7), 
k=1 

where 

(7.17) Cae = Ут. 
ісі 


Consequently, if the sums of the paired products с„ are first obtained, equa- 
tion (7.16) may be used instead of (7.12) to calculate the а, values. Hence 
the two iterations which were involved in obtaining a; (by multiplication 
with the correlations) are reduced to only one operation with the num- 
bers cs}. The number of iterations required for any degree of accuracy is 
thus cut in half. 

It will be found very convenient to employ matrix notation in dealing 
with complex expressions of the form (7.17). If R = |[rj;|| is used to denote 
the matrix of correlations, the c,; is the element in the vth row and kth 
column of the product of the symmetric matrix R by itself. The matrix 


* “Analysis of a Complex of Statistical Variables into Principal Components,” Sec. 4. 
1 Hotelling, “Simplified Calculation of Principal Components." 
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of these elements may be represented by С = ||с..|. It is, therefore, ap- 
parent that 


€ = [lel = |141 - 14| = R’ > R = Re А 


where the last equality follows from the fact that the transpose of а sym- 
metrie matrix is equal to the matrix itself. Substitution of the trial values 
ап, az... , anı in (7.12) is equivalent to multiplying them by R (і.е., 
obtaining the sums of the paired products with the elements of the col- 
umns of R). Substitution in (7.16), on the other hand, amounts to the mul- 
tiplication by R?, 

The improvement in the iteration process need not end with the employ- 
ment of R?. After doubling the speed of convergence by squaring R, it can 
be doubled again by squaring R?, i.e., by multiplying a set of trial values 
by R‘, and thus the equivalent of four multiplications by R is obtained. 
Upon squaring again, a matrix R5 is obtained, and multiplication by it is 
equivalent to eight multiplications by R, and so forth to any power of the 
correlation matrix. This squaring process is continued until the conver- 
gence is so rapid that additional matrix Squaring is not worth while. In 
Appendix D a scheme for determining the number of times a matrix should 
be squared is explained. 

The second and remaining principal factors may be determined by the 
same method, and the convergence can be accelerated by the use of a con- 
venient power of the matrix of residual correlations. Itisnot necessary, how- 
ever, to obtain this power of the residual matrix by repeated squarings, а8 
was done in the case of the original matrix of correlations. Instead, the de- 
termination already made of the power of R and the following algebraic 
properties of matrices can be employed for this purpose. 


Denoting the matrix of first-factor residuals by Ruy, it follows from (7.9) 
that 


(7.18) Ri = [lri] = Ilr; – апа =R — Q,, 
where 
Оз = |.| = (адан) (%-1,2...;7) 


is used to represent the matrix of products of first-factor coefficients. Now 
it is shown in equation (B.30) that 


n 


> Tikan = Ман 


j=1 


n 
> б == Әр; 
зі 


and in (7.11) that 


” 
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These lead to the following relationships among the matrices;* -= 


P = QR = МО, D 
01-701. 


Ву repeated use of (7.19) it can readily be shown that 


io = RQ = MQ;, 
Qi = AQ, 


(7.19) 


(7.20) 


where е is any positive integer. Hence employing (7.19), 

Rj = (R – 01) = R? -2RQ + Qi = RF — XQ, 
and, in general, 
(7.21) Ri = В°— NQ, . 


Thus the eth power of the residual matrix is expressed in terms of the eth 
power of the original correlation matrix, obviating actual squaring of the 
residual matrix. 

From the foregoing development the order of procedure of the iterative 
Scheme may be summarized. Using К as the basis of selection of the set 
of trial values, this set rapidly yields the values of the first-factor coeffi- 
cients} and the characteristic root №. Furthermore, the value M will be 
determined from the multiplication of the set of trial values by R*, and 
A! сап be obtained by divison. Then multiplying №! by each element 
of Qi and subtracting from the corresponding element of Ве, the eth power 
of the residual matrix is obtained. The second-factor coefficients are ob- 


* To indicate the derivation of (7.19), the algebra leading to the last relation will be 
given in detail. The square of the matrix Qi = |алаһ| is given by 


Qi = ||ала,. |01211 (7, 0 = 1,2,...,т),, 


іп which the jth row of the first representation of Q, and the kth column of the second 
representation are obtained by letting v range from 1 to n. Thus a representative row is 


Hanan апаз... апа,.|| and a representative column (written horizontally to save space) 
p (апан алак . . . алан). Row-by-column multiplication of the matrices then pro- 
uces 


= = Jlajaeral] = 1Q1. 


алап 5. а, 


кті 


Qi- | mananan 
peri 

T In “Тһе Unit Hierarchy and Its Properties" (Psychometrika, 1938), Burt has pointed 
out that to factor a matrix Бе is equivalent to obtaining a Spearman general factor. 
This arises from the fact that, with a sufficient number of self-multiplications, апу sym- 
metric matrix can be reduced as closely as desired to а matrix of rank опе. 
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tained from R; and В; in the same manner as the first-factor coefficients are 
determined from R and R*. It may not be necessary to employ the eth 
power of R, in the calculation of the second-factor coefficients when rapid 
convergence is evident. Then some lower power of В), or R; itself, is em- 
ployed. This is illustrated in Appendix D. To calculate the third-factor 
coefficients, the matrix Rs of the second-factor residuals and R$ (or some 
lower power of Re) are employed. The latter matrix is obtained convenient- 
ly by an expression of the form (7.21) relating the second- to the first-factor 
residuals. Further factors are determined similarly until approximately all 
the communality is analyzed. 


7.5. Numerical Applications 


1. EIGHT PHYSICAL VARIABLES 


The first illustration for the method of principal factors was obtained 
from Mrs. Mullen's data, which were referred to in 4.6. Her analysis of 
seventeen physical variables by the bi-factor method revealed a general 
size factor and two outstanding group factors identified with longitudinal 
and horizontal growth. After this solution was obtained, an alternative 
form, which might be preferred by the biologist, was suggested by the fact 
that the latter two factors might be considered as opposing measures of а 
single factor. A solution with a general physical growth factor and a bipolar 
factor* (representing the longitudinal versus horizontal growth) was then 
proposed. 

For the purpose of making such a principal-factor solution eight of the 
total set of variables were selected. Of the set of seventeen variables, three 
indices—of head size, hand squeeze, and lung capacity— were eliminated be- 
cause they were not measures of the two group factors. The following vari- 
ables were dropped because of small, or insignificant, group-factor coeffi- 
cients: sitting height, bi-iliac diameter, chest depth, and shoulder width. 
Тһе eight variables retained include two subgroups consisting of four longi- 
tudinal and four horizontal variables. The correlations among these vari- 
ables are presented in Table 7.1. 

In Mrs. Mullen's study the complete analysis was made by the bi-factor 
method which furnished the communalities for all variables. The portion 
of the bi-factor pattern for the eight variables, also including their commu- 
nalities, is given in Table 7.2. Thus estimates of the communalities of these 
variables for further analysis into a principal-factor pattern are immediately 
available. The communalities of these variables were also obtained directly 
from the correlations by the methods of Chapter IV. The latter values, 


* See 5.7. 
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which are given in Table 4.5, are compared with the bi-factor estimates of 
the communalities in Table 7.3. It is evident, from this comparison that 
there would be little practical difference between the final solution based 
upon these two sets of communalities. Since the communalities of Table 7.2 
were available at the time the analysis was first made, they are employed іп 


Appendix D. 
TABLE 7.1 


INTERCORRELATIONS OF EIGHT PHYSICAL VARIABLES 
FOR 805 FIFTEEN- YEAR-OLD GIRLS 


Variable 


dei pala деер» : .380 
. Bitrochanteric diameter.| . T .319 
> Chest gikthi. зла» .30 Е .237 
Обен Н „аена E 3 .345 


PND peN 
5 
9; 
E 


TABLE 7.2 


PORTION OF BI-FACTOR PATTERN FOR SEVENTEEN 
PHYSICAL VARIABLES* 


General anims 
Variable Physical | Lankiness | Stockiness > 
Growth pality 
2 А B e қ 
.691 .854 
.591 .897 
.581 ‚833 
.598 .783 
.694 .870 
‚611 .687 
.562 .521 
‚596 .579 


a Taken from Frances Mullen, "Factors іп the Growth of Girls Seven to 
Seventeen Years of Age" (Ph.D. dissertation, Department of Education, Uni- 
versity of Chicago), 1939, Table 27, p. 45. 

Тһе principal-factor pattern for the eight variables is presented in Table 
7.4. The adequacy of factorization is shown in several ways. The compari- 
Son of communalities in the last column of Table 7.4 shows that the values 
Obtained from the pattern differ only slightly from those originally em- 
ployed in the correlation matrix. Actually, the total calculated commu- 
nality (5.968) accounts for 99.1 per cent of the original communality (6.024). 
The completeness of factorization is also indicated in the process of com- 
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puting the respective factor coefficients. Thus the contribution of the first 
factor (i.e., the sum of the squares of its coefficients) is 4.455 or 74.0 per cent 
of the original communality. Similarly, the contribution of the second fac- 


TABLE 7.3 


COMPARISON OF COMMUNALITIES FOR 
EiGHT PHYSICAL VARIABLES 


a) | (2) 


Variable Calculated | Bi-factor | Difference 
Directly | Estimates | 72 
‚842 .854 | —.012 
‘881 | 897 —016 
.817 .833 — .016 
.815 .783 .032 
.872 .870 .002 
.647 .687 —.040 
.584 .921 .063 
.502 .579 —.077 
TABLE 7.4 


PRINCIPAL-FACTOR PATTERN FOR EIGHT 
PHYSICAL VARIABLES 


PATTERN СОЕҒҒІСІЕМТВЗ COMMUNALITY 
VARIABLE = 

n а) (2) = 
2 © d ч Original |Calculated = 
E Height... iuo vue xm .858 —.828 | .395 .854 .844 .010 
2. AIT T so ues cds Тем 849 —.414 .328 .897 .892 .005 
3. Length of forearm. . . .810 | —.412 417 .833 .826 .007 
4. Length of lowerleg......| .825 —.339 .452 .783 .796 —.013 
Б, WEICHE s сыл қанша ri dm ‚747 ‚561 ‚357 ‚870 .873 | —.003 
6. Bitrochanterie diameter..| .637 .507 .581 .687 .663 .024 
T GChest-girth. sss sos coasa .561 .488 | .669 .521 .558 —.032 
8. Chest width. ........... .619 -371 .692 .579 „521 ‚058 
ӘШІР; a otia ЫСЫ кан ыл заана 6.024 | 5.968 054 
Contribution of factor...... 4.455 ШОШ [ope ao Lorna sanas ooa Желе? 

Per cent of total original com- 
ТАШНА У rnana curis 74.0 Db. Cosas cime 99.1 9 


* Since the reliability of any one of these physical variables is close to unity, the unique factor in each 
case may be considered as essentially the specific factur. The index of completeness of factorization (2.14) 
is then approximately 100 times the calculated communality of each variable. 
tor is 1.511 or 25.1 per cent of the communality. In this example only two 
factors are required to account for practically all the common-factor vari- 
ance. Completeness of factorization is also evident from Table D.13 of final 
residuals in Appendix D. 
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The coefficients of the first factor in Table 7.4 are all large and positive, 
indicating an important general factor of physical growth (G) among these 
variables. On the other hand, the second factor has loadings of opposite 
signs for the two subgroups of variables. From the nature of the variables, 
this bipolar factor might be called "Stockiness." If desired, of course, the 
signs of all the coefficients of this factor may be changed. Then this factor 
might be labeled ‘“‘Lankiness.” 

Whatever name is selected for a bipolar factor, it should have a clearly 
recognizable negation. À more fundamental approach is to find a basic 
term which connotes the entire continuum. For example, a bipolar factor 
which is named “Heat” (or, Cold") would have the opposite characteristic 
“Cold” (or, Heat"). A пате representing both of these characteristics is 
“Temperature.” These two approaches may be indicated schematically as 
in Figure 7.1. Another example is a bipolar factor named “Fear,” the nega- 


a) Cold Heat 
0 
b) 
0 Temperature 
Fio. 7.1 


tive of which is clearly “Courage.” A fundamental term to describe this 
continuum, however, is difficult to determine. The investigator may have 
trouble in finding an appropriate name, of either type, for a bipolar factor 
because of the psychological difficulty of projecting the interpretation be- 
yond the immediate content of the data. 

Inasmuch as “Stockiness” and *Lankiness" are not clearly distinguish- 
able as opposites (according to a of Fig. 7.1), neither of these seems to be 
an appropriate name for the bipolar factor. In an attempt to get a name, of 
the type b, which transcends the specific descriptions of the variables, the 
term “Body Туре” (BT) has been adopted. On this continuum, variables 
describing different body types have projections of opposite sign. 

Geometric interpretations of the two factor solutions will now be given. 
In Figure 7.2 the coordinates of the points representing the eight variables, 
from Table 7.2, are plotted with respect to the three bi-factor axes. Thus 
the coordinates of the first point are (.691, .614, 0). It will be noted that all 
points have a zero coordinate for either B or C, and hence lie in one of the 
reference planes determined by the A and B axes or the A and C axes. It 
Should be observed that the points lie exactly in the reference planes only 
because they represent theoretical variables, in the sense of (2.4). The pro- 
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Jections of the first four points on the A and B axes are indicated by crosses, 
and the projections of the last four points on the А and C axes are given 
by circles. The cluster of positive projections on the A axis is the geometric 
basis for the naming of the general factor, while the clusters on the other 
two axes furnish interpretations for the group factors. 

The diagram for these eight variables in the plane of the two principal 
factors is presented in Figure 7.3, the coordinates being taken from Table 
7.4. The two subgroups of variables lie in the first and fourth quadrants. 


Fic. 7.2 


Hence the projections of all the points form a single cluster on the positive 
end of the G axis. The projections on the BT axis, on the other hand, fall 
into two clusters which are widely separated. The projections on the re- 
spective axes give the geometric basis for the naming of the general and bi- 
polar factors. 

The bi-factor analysis of the eight variables, as shown in Table 7.2, in- 
dieates а general physical growth factor А, a lankiness factor B, and а 
stockiness factor C. Such a solution is certainly a satisfactory description 
of the variables in terms of factors with positive coefficients. For these 
physical variables, however, the biologist might prefer an analysis in terms 
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of a general factor and another factor which expresses the two preceding 
factors simultaneously, as indicated in Figure 7.3. This preference might 
be based upon the opposing nature of the two groups of variables which 
makes it possible to conceive of them as measures in opposite directions on 
а common scale. In this example such preference would also lead to parsi- 
mony of description, inasmuch as the total number of common factors is 
reduced from three to two. The biologist would then choose the principal- 
factor solution of Table 7.4. 


BT 


Fio. 7.3 


These considerations as to choice of form of solution are in harmony with 
the principles set down in Chapter V. One of the most important bases for 
the selection of a preferred type of solution in any field of investigation is 
the nature of the variables. Statistical standards, such as complexity of 
variables or parsimony of factors, are of lesser importance in making a de- 
cision. The investigator is fortunate when the variables of his study lend 
themselves to a clear-cut choice of solution. In such a case, the final pattern 
could be considered almost unique, in the sense that other workers would 


also accept it as the preferred type. 
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2. TWENTY-FOUR PSYCHOLOGICAL TESTS 
The next illustration is based upon the twenty-four psychological tests 
of 2.8. Although it would not be expected that variables of this nature 
would ordinarily be interpreted in bipolar form by psychologists, it was de- 
cided to carry the analysis far enough to see the essence of a principal-factor 


TABLE 7.5 


FIRST Two PRINCIPAL FACTORS FOR 
TWENTY-FOUR Рвүсно- 


LOGICAL TESTS 
Variable G VR 

1 601 —.051 

‚ ҖЕ 371 .018 

8: 423 .097 

4.. 482 094 

5.. 690 .821 

6.. 683 ‚402 

ce 683 .449 

8.. 676 .206 

9.. .693 .449 

10.. .459 — .430 

11. ‚557 —.355 

12 471 —.492 

1 .599 —.259 

1 .430 —.072 

1 .393 —.115 

1 .505 —.110 

.474 —.238 

.523 —.348 

.445 -.118 

615 ‚124 

598 —.210 

612 .096 

692 .056 

652 —.158 

{асбот.... 7.657 1.669 
Per cent of total 
original com- 

munality....| 65.4 14.3 


solution. This implied the calculation of the coefficients for only the first 
two factors, for all the other factors would be of the form of the second. 
Table 7.5 presents this portion of the principal-factor pattern. 

The labor of computation in the direct principal-factor solution increases 
very rapidly with the number of variables and factors. For the example of 
eight physical variables, the total time for complete factorization was less 
than ten hours, indicating the feasibility of this method for such set of 
variables. In the case of the twenty-four psychological tests, however, the 


| 
| 
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time required for the calculation of the first-factor weights alone was more 
than seventy hours. Each additional factor would probably require up- 
ward of forty hours depending upon the number of iterations. In case а 
machine is devised which will simplify the type of multiplication involved, 
the direct principal-factor method will then be appropriate for large sets of 
variables. 

The interpretation of the two factors in Table 7.5 will now be given. Asin 
all cases of principal-factor patterns, based upon a set of positive correla- 
tions, the first factor has appreciably high positive loadings for every test. 
This factor might then be regarded as general ability (G). 

Тһе second factor may be named from the tests with significant coeffi- 
cients. As а rough estimate the standard error of a factor coefficient may 
be taken from Table H.1. For the present example this approximation to 
the standard error* is .109 so that the level of significance of .30 may be 
appropriate. Those variables which have significant coefficients, by this 


standard, are listed below: 


Variable Ccefficient 
5. General їїогтайоп................... .821 
6. Paragraph comprehension........... .402 
7. Sentence сотріейоп................... 449 
9. Word meaning....... өз ‚449 
10. Speed of аййїйоп.............. 5 — .430 
11. Speed of code translation .. —.855 
12. Speed of counting добв................. — .492 
18. Number-figure тетогу................ —.848 


The first four tests, which are verbal in character have positive correlations 
with the second factor, while the last four, which are essentially speed tests, 
have negative correlations with this factor. These subgroups of tests agree 
substantially with the grouping by the method of B-coefficients described 
in 2.9, (Although Test 18 was put in the memory group, it is also a meas- 
ure of speed of reaction.) It appears difficult to find the common element 
which underlies the content of these eight variables by which to name the 
factor. A tentative name which might be attached to this factor is “Verbal 
Rigidity” (VR). ү 
Subsequent factors will, of course, be of the bipolar form and might be 
given similar interpretations in terms of the subgroups of tests with signifi- 
cant loadings. Such subgroups may include some variables which were em- 
ployed in the naming of VR. Owing to this increased complexity of the 
variables, the naming of later factors may be more involved. A complete 
derived principal-factor pattern for these twenty-four tests is given in 9.8. 


* See p. 149. 
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3. EIGHT EMOTIONAL TRAITS 

Тһе particular appropriateness of the principal-factor method to another 
field of psychology will now be indicated. The material for this illustration 
includes eight emotional traits, the correlations for which were furnished 


TABLE 7.6 


INTERCORRELATIONS OF EIGHT EMOTIONAL VARIABLES 
FOR 172 NORMAL CHILDREN AGED NINE TO TWELVE 


Variable | 1 | 2 | 3 4 5 6 7 8 


. Sociability Чї lea aces ітес теа ен [Ока ade Oa A oe aS 
Sorrow.... 83 МЕСЕ dete inn ч 
81 387 || 4380 |за есенсе 
80 | .62 | .63 DO. [ашна eges 
71 59 | .37 49 BY. E pcs 


омотњоюн 


TABLE 7.7 
PRINCIPAL-FACTOR PATTERN FOR EIGHT EMOTIONAL TRAITS 


Common Factors Соммомлілтү 
VARIABLE 
a) (2) 
98 ® Original | Calculatea| (7? 
1. Sociability. . . .98 ‚06 .94 .96 —.02 
9; .95 14 .94 .92 .02 
3. .89 .92 —.08 
4. 50 .53 —.08 
5. 57 .56 01 
6. 28 .30 —.02 
7. 63 .63 00 
8. 12 .14 —.02 
4.87 4.96 —.09 
Contribution of 
factor. a isa ei 4.17 МӨ осалы ға | inn аға йа | e қылт өза 
Per cent of total 
original com- 
munality..... 85.6 16.2 |. газны 101.8 -1.8 


by Burt,* who has also discussed the factorial analyses of variables of this 
type in à subsequent paper.[ Although he does not furnish & principal- 
factor solution, he does point out the bipolar nature of emotionality factors. 
ж Cyril Burt, “General and Specific Factors Underlying the Primary Emotions,” Re- 
port of the British Association for the Advancement of Science, 1915, pp. 694-96. 
+ “The Factorial Analysis of Emotional Traits,” Character and Personality, 1939, PP- 
238-54, 285-99. 
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The traits which are analyzed here are indicated in Table 7.6, where the 
correlations are presented. The communalities were determined from а solu- 
tion essentially of the bi-factor form. It was found that only two principal 
factors were necessary to account for these communalities. The factor pat- 
tern is given in Table 7.7, where only the common factors are presented. 

The first factor may appropriately be called "General Emotionality" 
(GE), although the present sample of emotional traits is small. In the pres- 
ent example the method of naming the bipolar factor is similar to that em- 
ployed in the case of the twenty-four tests. An approximate measure of the 
standard error of a factor coefficient for N = 172 and an average correla- 
tion of .48 is .065 from Table H.l. The traits with significant. coeffi- 
cients are anger (.60), wonder (.32), and tenderness (—.51). Since wonder 
and anger are indicative of an egocentric personality, and tenderness is 
indicative of timidity, the factor characterizing these two opposing emo- 
tions may be called “Еросепігісіу” (E). If itis desired to change the signs 
of all the coefficients, then the factor may be called “Timidity.” In Burt's 
discussion fear and sorrow are classed with tenderness, and in the present 
analysis each of these traits has а coefficient of —.14. These values have 
some statistical significance and help substantiate the naming of the sec- 


ond factor. 
4. EIGHT POLITICAL VARIABLES 


The final example has been selected from a set of political variables in 
order to illustrate the applicability of the principal-factor solution in an 
entirely different field. The data also furnish а solution in which all the 
factors, including the first, are of the bipolar form. A set of eight variables 
was selected from a larger group of seventeen political variables, analyzed 
by Gosnell and Schmidt.* The smaller set, which was taken for simplicity, 
nevertheless includes the variables which are among the best measures of the 
factors given in Gosnell's solution. A brief description of these variables, 


measured in 147 Chicago election areas, follows: 


Lewis: Percentage of the total Democratic and Republican vote cast for Lewis 

Roosevelt: Corresponding percentage for Roosevelt 

Party voting: Percentage that the straight-party votes were of the total 

- Median rental: Median rental (in dollars) 

. Homeownership: Percentage of the total families that own their homes 

. Unemployment: Percentage unemployed in 1921 of the gainful workers ten years 

of age and over 

7. Mobility: Percentage of total families that have lived less than one year at pres- 
ent address 

8. Education: Percentage of population, eighteen years and older, which completed 
more than ten grades of school 
* Harold F. Gosnell and Margaret Schmidt, “Factorial and Correlational Analysis of 

the 1934 Vote in Chicago," Journal of the American Statistical Association, 1936, pp. 507-18. 


PoP ON 
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The intercorrelations of these variables are given in Table 7.8, in which 
communalities are also recorded. These communalities were computed by 
the second method of 4.6, in which grouping of variables is not employed. 
For example, the caleulation of hi was based upon the stub of Table 4.6. 


ommunality, four were dropped because 


TABLE 7.8 


INTERCORRELATIONS OF EIGHT POLITICAL 
VARIABLES FOR 147 ELECTION AREAS 


Roose- Party Median BOR Unem- Mo- Educa- 
velt Voting Rental Ыр. ployment 


Variable | Lewis 


ay be termed “Home Permanency”’ 
t three variables again appear 


is bipolar factor is conveniently de- 


THE PRINCIPAL-FACTOR SOLUTION 


179 


scribed by a single name because the opposing variables may be considered 
88 measures on a single scale in opposite directions. 

From the foregoing illustrations it is apparent that the direct principal- 
factor solution is well adapted to small sets of variables, say, less than 


TABLE 7.9 
PRINCIPAL-FACTOR PATTERN FOR EIGHT POLITICAL VARIABLES 
Common Factors COMMUNALITY 
VARIABLE 
(1) (2) 
j TOW са Original |Саїсымеа| 922 
-.28 ‚52 ‚55 —.08 
-.48 1.00 1.00 -00 
--.17 .78 .79 2401 
—.09 .82 .78 .04 
65 .36 .50 -.14 
01 .80 .79 01 
56 .63 75 —.12 
15 .97 94 .03 
тулк 5.88 6.10 —.22 
ү. ЖЕТЕР КИНИНИ! БЕТІ”, 
original com-| 
minality Bi wees 85.2 88. — кете 103.7 —3.7 


twelve. The direct method does not appear to be feasible when dealing 
with larger sets of variables because of the excessive labor involved. In 

hapter IX, however, an indirect method is presented for obtaining а 
Principal-factor pattern by rotation from some other form of solution. Thus 
if some factor solution can be simply obtained, it may be rotated to the 
Principal-factor form. By this procedure a large set of variables can be 
analyzed in the principal-factor form with only a fraction of the labor re- 


Wired by the direct method. 


СНАРТЕВ УШ 
ТНЕ CENTROID SOLUTION 
8.1. Introduction 


In the two preceding chapters methods have been presented for obtaining 
two of the preferred types of factor solutions discussed in Chapter V. А 
third type of preferred pattern, the multiple-factor solution, was indicated 
іп б.6. A direct method of analysis for this type of pattern is not available. 


` Instead, any factor pattern which has the property (2.24) of reproducing the 


correlations may first be obtained and then transformed into the multiple- 
factor form by the methods of Chapter X. The problem of obtaining a mul- 
tiple-factor pattern thus consists of two parts: first, the calculation of a 
preliminary pattern and, second, the transformation of this pattern to the 
multiple-factor type. In this chapter two such preliminary patterns are de- 
veloped. These patterns may then be transformed to any desired form of 
solution. In particular, the rotation to the multiple-factor form is presented 
in Chapter X. An illustration of the rotation from a preliminary pattern 
to the principal-factor form is also given in Chapter IX. 

The two preliminary solutions are the centroid and averoid patterns. The 
theoretical development of the first of these is presented in 8.2, and a list 
of steps for routine calculation is given in Appendix E. Three illustrations 
of the centroid pattern are presented іп 8.3. The averoid method of anal- 


ysis, discussed in 8.4, is similar to the centroid method, and, therefore, по 
additional outline for calculation is required. 


8.2. Centroid Method 


The fundamental formula of the centroid solution was first employed by 
Burt* in obtaining a single general factor of the Spearman form. Later, 
when the analysis of psychological tests shifted from one to several common 
factors, Thurstonef provided the complete form for the centroid method, 
which is followed in the main in the present development for the analysis 
of any set of statistical variables. 


Тһе name of the methód connotes its close relationship to the mechani- 

* Cyril Burt, The Distribution and Relations of Educational Abilities, 1917, p. 53. 

+L. L. Thurstone, “Multiple Factor Analysis,” Psychological Review, 1931, рр. 406- 
27; The Vectors of Mind, 1935, Chap. ІП. 
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cal concept of a centroid, or center of gravity. For this reason, the centroid 
form of analysis can best be described in geometric terms. According to 
Chapter III, the variables may be considered as represented by a set of n 
vectors which are contained in a space of m dimensions, where m is the 
number of common factors; and the scalar product of any pair of vectors is 
the correlation between them, as given in (3.51). The variables may also 
be considered as represented by the m coordinates of the end points of these 
vectors with respect to т mutually orthogonal arbitrary reference axes. 
Since the configuration of the vectors representing the variables completely 
determines the intercorrelations, the reference system may be rotated with- 
out any effect on them. The arbitrary coordinate system may then be ro- 
tated so that the centroid point of the set of n points, along with the origin, 
determines the first axis of reference. Thus it is possible to obtain the pro- 
jection of each of the vectors, or the coordinate of each variable, on the first 
axis of reference through the centroid. 

Starting with a factor pattern of the usual form (2.16), the correlations 
аге reproduced by means of equation (2.22) when the common factors are 
uncorrelated. Then, making the same assumption as in the preceding meth- 
ods of analysis, namely, that the residuals vanish, the observed correlations 


may be written 


(8.1) Tik = Tja = алам + Ajk +. F анат 1 
G, k-12,...,n), 


Where m is the number of common factors. The numerical values of а; 
@ = 1, 2,..., m) аге determined by the position of the orthogonal ref- 
erence axes, since a; is the tth coordinate of variable z;.* Now let the frame 
of reference be so selected that the first axis Fı passes through the centroid 


of the system of n points: 
Р, (ац, 0... 01m) » Palan, азу... 00m) 5) 
Р„(аы, йһ›..+›@вт). 
Any one of the m coordinates of the centroid (say, the first one) is the aver- 


age of the corresponding coordinates (the first ones) of the n points. Thus, 
1n general, the tth coordinate of the centroid is given by 


(8.2) Su (:-1,2,...,т), 
k 


* More precisely, this variable should be designated by 2; since it is represented in 
] 


the common.factor space. See 3.8. 


e 
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where the summation is from 1 to n on the indicated index when the limits 
are not given specifically. In the present case the centroid lies in the first 
axis of reference, so that its coordinates are all zero except the first, that is, 


(8.3) Уа = уан = ... = Sain = 0 
k k k 


Тһе m values (8.2) then reduce to 


1 
атқ Dh ios = 


there being (m — 1) zeros. Since the centroid lies in the first axis, the first 
coordinate is also the distance of the centroid from the origin. 

It is now possible to determine the coefficients of the first centroid fac- 
tor, i.e., the coordinates ад, in terms of the observed correlations. For, sum- 
ming (8. 1) for all variables Ё in a fixed column j of the correlation matrix, 
there results 


Sr = an( Seu) + on( Zan) +...+ o (Xo) А 
k E n 


k 


which, on applying (8.3), becomes 


(8.4) У = (Жа) " 
E k 


Then the sum of all the entries in the correlation matrix is simply 


eo EX» 


k 


Now substituting + %> хх» 2» for x in (8.4), and arbitrarily taking 


the positive sign, the кур. байн for aj - be written explicitly as follows: 
ку 

No MM S; 

js * 

2 [4 


(8.6) 
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where S; is the sum of all the correlations in column j of the correlation 
matrix, and T' is the total of all the correlations in the matrix, including the 
diagonal terms in both of these sums. Of course, if the negative sign of the 
radical had been chosen, the coefficients of the factor would all be changed 
in sign, yielding an equally acceptable factor. Formula (8.6) gives the coeffi- 
cient of the first centroid factor Р, for cach variable z;, or the first coordinate 
for each point representing a variable. 

The next step is to get the first-factor residuals, from which the second 
coordinates are found. Since the residual correlations with one, two, ..., 
(m — 1) factors removed are employed in successive stages of the centroid 
method, the notation introduced for the principal-factor method in 7.3 will 
be employed again. The first-factor residuals are written 


(8.7) Wyk = Tj — алак = алак + ajaks +... + анат. 


The residual correlations may be regarded as the scalar products of pairs 
of residual vectors in a space of (m — 1) dimensions—the dimension of the 
residual space being equal to the number of terms in the right-hand mem- 
ber of (8.7), or the rank of the matrix of residual correlations, according to 
Theorem 3.5. 

In the (m — 1) residual space, the n points representing the variables 
have the following sets of coordinates: 


Pi(015 Qis, . ~~ , Qim), Р»(ав», 023, ... ат)... 


Polana; Gus; «э чу бит)» 


Then the (m — 1) coordinates of the centroid of these n points are 


E ам (10:29, naM) y 


Which all vanish according to (8.3). Thus the centroid is at the origin in 
this (m — 1) space, and formula (8.6) cannot be used directly for calculat- 
ing the values of the second-factor coefficients. It will be noted that in ob- 


taining (8.6) there was involved the division by Sau, or n times the dis- 


D 
tance of the centroid from the origin. It was tacitly assumed that the cen- 
troid was not at the origin, for otherwise this division would not have been 
Possible, © 

The immediate problem, then, is to remove the centroid from the origin 
of the (m — 1) space, so that the preceding method can again be applied. 
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By means of rotations of certain of the vectors about the origin through 
180?— also called reflections in the origin—the centroid can be removed from 
the origin. If the coordinates of a point P, representing а variable 2;, are 


(ал, бю...) бін)» 
then the reflected point. — P;, with coordinates 
(—ал, -а»...» 70m), 


represents the variable —z;. Such a variable may be interpreted as the 
original variable measured in the opposite direction. 

Now it is evident from (8.1) that to reverse the signs of the coordinates 
of P; has the effect of reversing the signs of all the correlations of variable z; 
with the remaining ones. Thus the reflection of a variable in the origin is 
accomplished merely by changing the signs of the correlations of this vari- 
able in the correlation matrix. Of course, the same argument holds for the 
residual (m — 1) space as for the original common-factor space of m dimen- 
sions. Hence the reflection of a variable in the residual space is accomplished 
by changing the signs of the residual correlations (8.7) for that variable. 

In an attempt to determine which variables to reflect, Thurstone* sug- 
gests that “it is desirable to account for as much as possible of the residual 
variance by each successive factor." It should be observed that the same 
idea is involved in the principal-factor method, where the maximization is 
accomplished by а mathematically rigorous procedure.f For the present 
case the maximization principle indicates that the second reference axis 
should pass through a cluster of residual vectors. If there is a clustering ofa 
set of residual vectors (i.e., a group of variables having high positive residual 
correlations), which is balanced by a scattering of vectors on the opposite 
side of the origin, since the centroid is at the origin, then the second refer- 
ence axis should be made to pass through this cluster. Thus it would seem 
that the vectors which scatter, opposite to a cluster, should be reflected so 
as to fall in with the group, and then the centroid of the n points will be in 
this group, and the second axis can pass through it and satisfy the foregoing 
principle of contributing to the variances as great a total as possible. The 
second centroid coordinates can then be computed as in the first case. In 
application, those variables which have the greatest number of negative 
correlations would be reflected first, bringing them into the hemisphere of 

* The Vectors of Mind, p. 96. 


{ A further analogy between the centroid and principal-factor methods may be noted. 
Each centroid factor is actually the first approximation, in the iterative scheme of 7.4, to 


the principal factor. 


| 
| 
| 
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the cluster. For practical problems, Thurstone suggests reversing ‘‘the signs 
of one trait at a time until the number of negative coefficients in the residual 
table is less than n/2,"* that is, less than n/2 negative signs for any one 
variable, not the entire table. One need not stop with this, however, for, 
if it is desired to further “maximize” the variance removed by each suc- 
cessive factor, the reflections of variables may be continued until the sum 
оа residual correlations for each variable is as large (positively) as pos- 
sible, 

For the remainder of the analytical work it will be convenient to use a 
symbol to designate whether a point, representing a variable, has been re- 
flected in the origin. Let ¢; stand for the algebraic sign of P;, that is, the 
point «P; is +P; or —P;. If P; has not been reflected, e; is plus, but, if 
P; has been reflected, then e; is minus. Furthermore, e; may be considered 
as an algebraic operator defined as follows: 


_ f +1if z; has not been reflected, 
€ 7 _ 1 if z; has been reflected. 


Then ¢; can be attached to the coordinates of P, and it can be treated as 
any other algebraic quantity. Thus, if the first-factor residual correlations 
after reflection of certain variables are designated by ту. (in distinction to 
1" jx before reflection), then they may be written as follows: 


(8.8) Тук = €;€4(Qjo0k2 + аузйкз ss + атат) . 


This result is immediate from (8.7) where each аҙ and ax; was replaced by 
ва and erary respectively, and then e; and e; were factored out algebraical- 
ly. If neither z; nor z, was reflected, or if both variables were reflected, 
then ту, = jrj; but, if only one or the other of z; and 2, was reflected, then 
Tijk = угда. In other words, Tijk = өкін). 

The (m — 1) coordinates of the centroid of the n points, after reflection 
of variables, are given by 


1 
a2, (502, 8, гау 00% 


Now the system of reference can be rotated about the first axis F, во that 
the second axis F; passes through this centroid.[ Let it be assumed that 


* The Vectors of Mind, p. 97. 

+ The residual (m — 1) space is orthogonal to the first axis of reference F;. The second 
axis F^, i.e., the first one in the (т — 1) subspace, тау then be rotated to any position 
іп the residual space, and it will be at right angles to Fi. 
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this has been done, there being no need to change the notation for the co- 
ordinates. Then the coordinates of the centroid are 


1 
s e nien By 


since the centroid lies in the axis Ёз. From the values for the last (т — 2) 
coordinates the following useful expressions may be written: 


(8.9) Mean = Увам =... = Уеа = 0. 
Е k k 


Now the projections of the vectors on the second centroid axis can be 
expressed in terms of the residual correlations. For, summing (8.8) for all 
variables k in a fixed column j of the residual correlation matrix (after re- 
flection of variables) and applying (8.9), there results 


(8.10) Mns = ваз > аав. 
n 


Е 


Then, summing for all columns, 
2 
(8.11) 2. Drie = Уа X aare = (Xs) : 
7 k j k k 
From (8.10) and (8.11) it follows that 
= Tijk 
D ба 
4 2, Drin Ут, 
7 k 


or, multiplying both sides by є, 


6032 


NET 


(8.12) а» = VAR G= 1,25: He 


where Sj is the sum of all the entries in column ј of the matrix of first-factor 
residual correlations and Т, is the total of all the correlations іп this matrix, 
the signs of all the entries being those after reflection. The e; indicates that, 
if the variable z; was reflected, then the algebraic sign must be changed; but, 
if the variable was not reflected, then e; is merely +1. In other words, 
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€jSj is the sum of all residual correlations for the unreflected variable z; 
with all other variables. Hence, by defining 


(8.13) 18; = 6 Si, 


formula (8.12) may be put in the form 


(8.14) аз = em (j51,2....,4). 


In this formula the numerator refers to the sum of the residual correlations 
for the unreflected variable j, while in the denominator 7; still stands for 
the sum of all residual correlations after the sign changes. Formula (8.14) 
gives the coefficients of the second centroid factor Р for each variable. 
Тһе remaining factor weights can be obtained in а similar manner. In 
general, the residual correlation of r;; after p factors have been removed is 


given by 


р 
(8.15) тал Vik — ` аййы = арар Ж... F атак 


t=1 


The centroid of the n points in this residual (m — p) space usually will be 
near the origin. Certain variables may then be reflected in the origin, begin- 
ning with the one which has the largest number of negative residual corre- 
lations. Thus, employing the same principle as before, the centroid may be 
removed from the new origin, and the factor Fp+ı determined so as to con- 
tribute to the residual variances of the variables as great a total as possible. 
After reflection of variables, the residual correlations become 


(8.16) Tyjk = €ji(ori) = Epik; 


Where the plus sign holds if neither 2; nor 2, was reflected, or if both were 
reflected, and the minus sign holds if only one or the other of the two vari- 
ables was reflected. Then, proceeding as before, the projections of the vec- 
tors on the (р + 1)st centroid axis, i.e., the coefficients of Ға, can be de- 
termined. They are given by 


(8.17 КРЕСТ ЖЕРГІ (4% 1 
) Tin ут, WT, p-0 


Where S;, is the sum of all the entries in column j of the matrix of residual 
correlations with p factors removed, and T, is the sum of all the correla- 
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tions in this matrix, the signs of the entries being those after reflection of 
variables. The definition (8.13) is extended to the general case, that is, 


(8.18) pj = єў = tSp; 


where the column sum 8; is not changed in sign if z; was not reflected, and 
changed if it was reflected. Formula (8.17) gives the coefficients of any 
centroid factor +1 for each variable. 

It is evident that formulas (8.6) and (8.12) are special cases of (8.17) for 
p=0andp= 1, respectively. The same type of formula is used to cal- 
culate the coefficients of the successive centroid factors. Since a basic prin- 
ciple of the centroid method is to account for as much as possible of the 
variances of the variables by each factor, the variables are reflected in the 
residual subspaces to bring them into a cluster. When the sign changes have 
been made, the centroid of the system of points in the residual space lies 
somewhere in the cluster of variables, and the next reference axis is selected 
through this centroid. Each successive centroid axis is at right angles to 
every one of the preceding axes because the residual space is orthogonal to 
the space of the centroid axes already established. Upon extracting each 
centroid factor, the residual correlations are reduced in magnitude, and the 
rank of each residual matrix is reduced by one. The foregoing development 
was made without any restrictions on the diagonal elements. The number 
of factors ultimately obtained is dependent upon these diagonal values, 
leading to the question of “when to stop factoring.” 

Thurstone* recommends that the largest correlation in each column of 
the correlation matrix be selected for the diagonal element of that column. 
In each subsequent residual matrix the calculated diagonal term is not re- 
tained but replaced by the largest residual correlation, regardless of sign, in 
each column. This procedure does not furnish a standard for determining 
the number of common factors. 

If, instead of modifying the diagonal entries at each stage, the analysis 
is applied directly, keeping the diagonal terms as calculated, then the num- 
ber of factors is determined when the original diagonal values have been 
completely resolved. Only the common factors will be obtained if the corre- 
lation matrix contains communalities in the principal diagonal and these 
values are completely analyzed. In practice, it is recommended that ap- 
propriate estimates of the communalities, discussed in 7.2, be employed. 
When such estimates are used and the straightforward centroid analysis 18 
applied, the resulting solution terminates in a definite number of common 
factors. This feature is illustrated in the next section. 


* The Vectors of Mind, Appen. 1. 
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8.3. Illustrations of Centroid Method 


1. THIRTEEN PSYCHOLOGICAL TESTS 
The first illustration of the centroid method is based upon the reduced 
Set of thirteen psychological tests of 6.11. The intercorrelations of these 
tests are given in Table 2.2 and are repeated in Appendix E, where the 


TABLE 8.1 
CENTROID PATTERN FOR THIRTEEN PSYCHOLOGICAL TESTS 
ComMon-Factor Coe 
COEFFICIENTS 
Test® | 1 
| " a) (2) 
er po wm % Original | Caloulatea | (0-0) 
1 607 —.060 —.443 | .558 .568 | —.010 
2 | 355 ‘038 | —.266 .203 .198 1005 
3 418 1148 | —.429 -362 ‘381 | —.019 
4 | 478 083 —.287 1314 .318 | —.004 
5 | 729 |  .257 1944 .646 657 | —.011 
6 | 707 `354 1167 ‘641 653 | -.012 
7 721 1367 1257 1750 721 “029 
8 705 1197 062 1571 540 031 
9 698 1409 252 2758 718 .040 
10 1455 | —.482 399 1554 599 | —.045 
H 1537 | —.390 145 1449 461 | —.012 
12 1487 | —.553 033 1531 544 | -.013 
13 |j 1674 | —.368 | —.135 1599 608 | -.009 
от EE à aes 6.936 6.966 | —.030 
Contribution of! 
factor....... | 4.620 1.392 КИ ЕДЫ 
Рег cent of total | | | | 
original com-| 
munality....| 66.6 | 20.1 13.8 pee 100.4 —0.4 
| | I 


* The names and brief descriptions of the tests appear in Appen. В.1. 


complete centroid analysis of these data is exhibited. According to the dis- 
cussion of choice of communality in 7.2, it is desirable that "complete esti- 
mates" be employed. As the best available estimates, the bi-faetor commu- 
nalities of Table 6.12 are used in the centroid analysis. 

Тһе centroid pattern for these thirteen tests is presented in Table 8.1, 
Where the centroid factors are denoted by C's and the unique factors have 
been omitted. Inasmuch as this solution is regarded as a preliminary one, 
the three common factors will not be named. Nevertheless, the common- 
factor Space is definitely determined by this solution. The adequacy of this 
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solution is indicated both by the magnitude of the final residuals, given in 
Table E.10, and by the degree to which the original communalities are 
analyzed. The comparison of the calculated and original communalities for 
each test, which is given in the last three columns of Table 8.1, shows that 
the factorization is quite adequate. The total calculated communality ac- 
counts for 100.4 per cent of the original communality, indicating a slight 
overfaetorization. The stage to which the factorization should be carried is 
also evident in the actual process of calculation, for, after each factor is ob- 
tained, the cumulative contributions of the factors can be compared with 
the total original communality. Thus, in the example, the first factor ac- 
counts for 4.620, or 66.6 per cent, of the communality, the first two factors 
account for 6.012, or 86.7 per cent, of the communality, while the three com- 
mon factors account for 6.966, or 100.4 per cent, of the total original com- 
munality. By all standards, it is clear that three common factors are suffi- 
cient to describe the given data. 

The form of this solution is very similar to that of the principal-factor 
pattern, including general and bipolar factors. The.centroid pattern might 
thus be retained as a final preferred type of solution. As indicated in the 
preceding section, however, the maximization of the contributions of the 
factors is mathematically more rigorous in the case of the principal solution, 
so it would generally be preferred as a final form. It will be shown in the 
next chapter that this more desirable form can be simply obtained by 8 
transformation from some preliminary solution, such as the centroid. 


2. TWENTY-FOUR PSYCHOLOGICAL TESTS 


Тһе centroid pattern for the twenty-four psychological tests of 2.8 is 
given in Table 8.2. This pattern may be shown to fit the original data Бу 
the standards employed in the preceding section. A multiple-factor solution 
will also be obtained in Chapter X by rotation of this centroid pattern. 

This example gives a practical illustration of the nature of the ‘‘conver- 
gence” to a relatively small number of common factors when complete 
estimates of the communalities are employed. As already noted, the solu- 
tion of Table 8.2 is statistically adequate as judged by the magnitude of the 
fourth-factor residuals and also by the fact that these factors account for 
over 97 per cent of the total original communality. To check the adequacy 
of this solution still further, the coefficients of a fifth factor have been caleu- 
lated and are presented in Table 8.3. It will be observed that except for 
one value all the coefficients are insignificant, so that the factor may be 
dropped. This example indicates how an analysis based upon complete esti- 
mates of the communalities shunts to a fixed number of factors. 
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TABLE 8.2 
CENTROID PATTERN FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 
COMMON-FACTOR COEFFICIENTS COMMUNALITY 
Test? 
а) (2) Ж 
бі За ө Ce | Original {Calculated @)-@) 
~.116 | „800 | —.250 .981 .536 .045 
—.119 .207 | —.135 .209 .214 | —.005 
—.220 .262 | —.155 .390 .323 .067 
m TT 1206 | —.184 815 1848 | —.033 
—.306 | —.344 .108 .668 .670 | —.002 
—.837 | —.258 .216 .643 .664 | —.021 
-.396 | —.384 .124 .786 .745 .041 
—.225 | —.153 | —.060 .530 .516 .014 
—.394 | —.240 .308 .708 .749 | —.041 
.455 | —.365 | —.136 .503 .572 | —.069 
.397 | —.208 | —.063 .500 „529 | —.029 
.360 | —.149 | —.388 575 .537 .038 
.130 | —.099 | —.402 .524 .558 | —.034 
.199 | —.013 .293 .448 .821 .127 
170 .146 .266 .350 .287 .063 
077 .300 076 871 875 | —.004 
317 .082 338 .551 .464 .087 
307 .248 072 .484 .460 .024 
125 .129 111 .273 .249 .024 
-.174 .128 004 .415 .421 | —.006 
114 .080 | —.171 .416 .410 .006 
—.144 .145 136 .415 ‚480 | —.015 
—.164 .129 | —.116 .539 .584 .005 
151 | -.150 | —.003 .507 473 084 
сены sinas aati ad ағ ырас 11.701 | 11.385 .316 
1.600 1.145 1,001 ШЖ е те tz Буе pw eee 
Рег cent of total i 
original commu- 
Пау. esie ei vee 65.3 13.7 9.8 826. . [eo 97.3 2.7 


“Тһе names and brief descriptions of the tests appear in Appen. В.1. 


TABLE 8.3 


FIFTH CENTROID FACTOR WEIGHTS FOR TWENTY-FOUR 


PSYCHOLOGICAL TESTS 


048 | 


oT OD 
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3. EIGHT PHYSICAL VARIABLES 
Ап additional illustration of a centroid solution is given in Table 8.4 for 
the physical variables of 7.5. This example is included to furnish a simple 
ilustration of a transformation from a centroid to a principal solution, 
which is given in the next chapter. Since the direct principal-factor solution 
is already available in Table 7.4, a comparison of these two methods of ob- 
taining such a solution can then be made. 


TABLE 8.4 
CENTROID PATTERN FOR EIGHT PHYSICAL VARIABLES 


Соммох-ЕлАстон с sim 
COEFFICIENTS адын 
VARIABLE 

қ Т @ | (2) B 

Gi Өй Original Calculated | (9-6) 
T. ЛИ ccs og axon хз», .830 -.896 | .854 | .846 .008 
2; Arm арап,.,...,....".. .818 —.469 |  .897 | .889 .008 
3. Length of forearm. ....| 777 —.470 | .83 | 825 .008 
4. Length of lower leg... . .798 —.401 | .783 | „798 —.015 
5. Weight: csc su ssi in ч... ‚786 .500 | 870 .868 .002 
6. Bitrochantericdiameter.| —.672 .458 | .687 | 661 ‚026 
7. Chost ш Т... e os ‚594 444 | ‚521 | 550 —.029 
8. Chest width........... .647 .333 .579 | .529 .050 

ШОТЫ; sii НИНЕН REC аны | 6.024 | 5.966 .058 
Contribution of factor....| 4.439 SERO. de oco aee D — 
Per cent of total original) | 
communality..........| 73.7 MBA ема | 99.0 1.0 


8.4. Averoid Method 


It has been seen that estimates of the communalities are required in the 
principal-factor and centroid methods of analysis. In this section a method 
is developed which may be employed when good estimates of the commu- 
nalities are not available. This situation may arise when the original design 
of the variables fails to produce intercorrelations that clearly indicate dis- 
tinct subgroups of variables. | 

It will now be shown that, if the average correlation for each variable is 
employed as the estimate of its communality in formula (8.6), a simplifica- 
tion of fhe method of analysis results. The sum S;in this formula, where the 
average correlation is employed for туу, is given by 


1 " 
(8.19) 8; = ( Фат) (гу; kAj К-1,0;... 2% 
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or, denoting the sum on the right by S/, 
(8.20) 8н СУ) 8; (1,91... ань 
The total Т of formula (8.6), with average correlations in the diagonal, 
reduces to 

Та-2(8,27-1,2,...) 0%) 
To) (а; jAk, 8-1,2 

(8.21) = (51) а) GK, jk-12,...,n) 


-(ың)т. 


Under the foregoing assumption, the first-factor coefficients are given by 


(2 )s 
5 .-1/% T n 
(8.22) aj, = E: : S! ls “DF 


(б-1,2,...,т), 


Where the prime is used on a; to distinguish it from the general value given 
by formula (8.6). 

The simplification of (8.22) over (8.6) is due to the fact that diagonal 
values are not explicitly involved. The values obtained by use of (8.22) are 
identical, of course, with the values which would be obtained by means of 
(8.6) with average correlations іп the diagonal. The expression (8.22) can 
also be shown to be equivalent to the ratio of the average of the (n — 1) 
Correlations in 5; to the square root of the average of the n(n — 1) corre- 
lations in Т”, Inasmuch as averages are employed in (8.22), the analysis 
based upon this formula is called the averoid method. 

After the coefficients of the first averoid factor have been obtained, the 
first-factor residuals are calculated. Certain variables are then reflected, 
as in the usual centroid method, in order to increase the contribution of the 
Second factor to the residual variance. Then a formula of the form (8.22) 
is applied to the residual correlations for the calculation of the second-factor 
Coefficients. This process is continued until the final residuals are considered 
as insignificant. In contrast to the centroid procedure discussed in 8.2, at 
each stage of the averoid analysis the calculated communalities are re- 
placed, in effect, by the new average residual correlations, 
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Inasmuch as precise standards for "when to stop factoring” are not 
known for such solutions, some crude basis must be employed. One such 
basis might be the numerical magnitude of the factor coefficients at а given 
stage of the analysis. If the largest of these values are considered to have 
little significance, then the last factor obtained would be rejected as negli- 


gible. 


8.5. Illustrations of Averoid Method 


To illustrate the averoid method, the thirteen psychological tests are 
again employed. It has been seen that very good estimates of the commu- 
nalities are available for this set of variables, and a centroid solution de- 
pendent on such estimates is given in Table 8.1. The centroid pattern based 
on the complete estimates of the communalities will therefore serve as & 
check on the adequacy of the averoid solution. 


TABLE 8.5 


CALCULATION OF THE A; COEFFICIENTS FROM THE INTERCORRELATIONS 
OF THIRTEEN PSYCHOLOGICAL TESTS 


527 619... eene 
i 1685! .539].....|..... 
1203 1246, .285| .170..... 
| 1353, .282 .300 .280) . 
|7095 .181| .271| .113| .585| . 

1309.345 .395 1280 408. 


4.709 4.705 4.769 4.527 2.8893. 
| .691| .690, .699 684 -424| - 


:480 .239| .321 


| | 


1..4.0342.484. 2.8063. 
аһ..| .592| .364 .411 


The intercorrelations of the thirteen tests are given in Table 8.5. The 
sums S/ of the correlations of each variable with all the others are recorded 
in the table. The total T" of all the intercorrelations is merely the sum of 
all the 8). Then the radical in formula (8.22) becomes 


13 
“/12(50578) - 4694, 
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and, upon multiplying this number by each sum Sj, the corresponding 
coefficient ал of the first averoid factor A; is obtained. 

The first-factor residuals are calculated by the same procedure as outlined 
for the centroid method in Appendix E and are presented in the lower half 
of Table 8.6. It will be noted that more than half of the signs in this part 
of the table are negative, and hence a factor determined from these 
residuals would be inappreciable. In order to increase the contribution of 
the second averoid factor, certain tests are reflected in the origin. The num- 
ber of negative residuals for any test is reduced to six or less by the methods 


TABLE 8.6 
FIRST-FACTOR RESIDUALS AND THE CALCULATION OF THE А; COEFFICIENTS 


2 3 4 5 6 7 8 9 -10| -11| —12 | -із 


VET VI EE 


for reflection outlined in Steps 11-26 of Appendix E. The residuals after 
such reflections are recorded in the upper half of Table 8.6. Тһеп the sec- 
ond-factor coefficients are calculated by means of the formula 


i n 
(8.23) af, = 68/14 lan: 


Where S/, is the sum of all the first-factor residuals for variable j, and Ty is 
the total of all the residuals, the signs being those after reflection. The sym- 
bol є; again indicates that if variable z; has been reflected then the algebraic 
Sign of the factor coefficient is changed. The sums S/, and the factor coeffi- 
cients a£, are also given in Table 8.6. 

The second-factor residuals, before and after reflection, are presented in 
Table 8.7, Then, applying a formula like (8.23), the coefficients а}; of the 
third averoid factor Аз are obtained. The residuals with the third factor 
removed are given in Table 8.8. Since the number of negative residuals for 
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any variable is less than six, no reflections were made in calculating the 


coefficients а}. 
In contrast to the magnitude of the third-factor coefficients, all but one 
of the a/,'s are very small. The largest of the latter values is only аң = 


TABLE 8.7 
SECOND-FACTOR RESIDUALS AND THE CALCULATION OF THE А; COEFFICIENTS 


Vari- * | | "mw 
Ver | -1 | -2| -з ii | | 6 | 7 8 | 9 | 10 | 11 12 | =13 
1 104| 100 
2 021| : 
Be 077 
4 137 
Bi | 
6. ; 031! 
7. 0б8 
8. 024:- 
9 144| 
10 119) 
11» 069, 
12. 022 - 
18. 032 — 
Sja 806! 
af 278) 


TABLE 8.8 
THIRD-FACTOR RESIDUALS 
(Final Residuals) 


= ; i i 
Vari- | | 5 | | | 13 
able | 1 2 3 | 4 5 6 7 | & | © | 10 11 12 | 
1 | | [ | ! 

2.. | 
3 | | 
4 | | | 
ы | 
6 025 : | | “| 
7 000: 058 | | | | 
8 000 — 048 040 | | | | 4 
9. 070 056 012- 036 | 

10. 003 - 0201-0122 039 — 057 ! | 

1 014 091 — 048'— 006 030 012 | 

12.. 000 — 049] 027] 047- 008 .I38 .— | 

13 010. 003 042 016 007 007 | | 022 

| | ms =н Ц 7 

Sj 321! 106} 255 156 097] 132 128 088 130 133| 118) 247 | 275 

аң 226 075, 180! 110 068 ооз, 090 062 094 | .083| 174 | 94 

| | | | ) = 


226, It is then clear that the fourth factor may be rejected as negligible. 
This, of course, is also evident from the magnitude of the third-factor residu- 
als, all of which appear to be insignificant. The first three factors are then 
retained in the final averoid pattern, which is exhibited in Table 8.9. 
Further tests of the adequacy of this solution may also be considered. 
The reliabilities of these psychological tests may be compared with the cal- 
culated communalities in order to check whether the factorization is in har- 
mony with the assumptions of 2.3. This check is afforded by the index of 
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factorization (2.14), which is given for each test in Table 8.9. It will be 
noted that only one index of factorization, namely, Hs = 100.3, slightly 
exceeds 100 per cent. Hence this standard indicates that the factorization 
has not been carried too far. The preceding test, however, showed that three 
factors were quite sufficient. Taken together these standards imply that 
three factors, and no more, are required to give an adequate description of 
the original data. 


TABLE 8.9 
AVEROID PATTERN FOR THIRTEEN PSYCHOLOGICAL TESTS 
CALCULATED INDEX оғ 
Common Factors | Cossu- PEL FACTORI- 
NALITY аы ZATION 
Test 

В 
А Аа Аз Aj ry Hj =100 22 
2 ! ті 

«каса, 592 .006 —.371 .488 756 64.6 

| 364 .016 -.250 ‚195 568 84.3 

411 ‚115 -.875 823 544 59.4 

484 ‚068 —.265 .309 922 88.5 

715 ‚264 .278 658 808 81.4 

691 867 ‚203 ‚658 651 100.3 

690 358 .277 .681 754 90.3 

699 205 .086 .938 680 79.1 

‚664 395 .266 .668 870 76.8 

| .424 —.419 .416 .528 952 55.5 

.530 — .393 .198 .475 712 66.7 

.463 —.523 .078 .494 .937 52.7 

| .660 —.364 —.078 .574 .889 64.6 

Contribution of, 

СЕО... uui. 4.391 1.285 POSE |І; TERES 


Тһе averoid solution of Table 8.9 may now be compared with the centroid 
Solution of Table 8.1. The corresponding coefficients in these two patterns 
аге very much alike, and the caleulated communalities of the respective 
Variables are not appreciably different. The total contribution of the three 
centroid factors is just 0.351 greater than the contribution of the three aver- 
oid factors. Thus the averoid solution accounts for all but 5 per cent of the 
total centroid communality. These discrepancies are due to employing aver- 
4ge correlations (in effect) at successive stages of analysis in place of com- 
Plete estimates of the communalities. With larger sets of variables, it is to 
be expected that these discrepancies will be even smaller. 

Evidently, one would prefer the centroid solution based upon complete 
estimates of the communalities if such a solution was feasible. The real 
need for a solution such as the averoid becomes apparent when an analysis 
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is attempted of a set of variables for which good estimates of communalities 
are difficult to determine.* 

Another illustration of the averoid solution is given in Table 8.10 for the 
complete set of twenty-four psychological tests. The agreement of this solu- 


TABLE 8.10 
AVEROID PATTERN FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 


Carcu- INDEX OF 

Соммох FACTORS РЕП Rer | picronr- 

Соммо- BILITY ZATION 

VARIABLE NALITY 

Di 
Р Ay As Aa Ai DH ел Heo. 
.596 | — .143 .273 | — .235 | .505 756 66.9 

‚876 | = .127 .210 | — .190 | :238 568 41.8 

1491 | — 1237 .250 | — .055 | .299 | .544 | 55.0 

479 | — .940 .216 | — .103 | .344 | .922 37.3 

68 | — 1285 | — 345 112 | .639 | 808 79.1 

.648 | = 811 | — 2256 1255 | .647 | .651 99.4 

.626 | — :366 | — 1359 :200| :695 | |754 92.1 

.658 | — :939 | — `166 .014 | 1518 | .680 76.2 

.646 | — :353 | — 1226 :312 | .690 | 1870 79.4 

449 -401 | — :398 | — 272 | `591 952 62.1 

562 -335 | — .236 | — :069 | “489 712 68.6 

466 .242 | — 1113] — ‘452 | “493 937 52.6 

601 -113 | — .092 | — {387 | ‘532 | 1889 59.9 

(431 :198 .022 .351| :349 | .648 53.8 

1402 1199 “110 :275 | 289 | 507 57.0 

1524 -083 | 293]  .091| 1376 | .600 | 62.7 

1476 1830 ‘018 ‘242 | :394 | 725 54.4 

1539 "310 226 | — (004 | .438 | 610 | 71.8 

1456 134 1115 .017 | 239 | 569 42.1 

1614 | — 140 1127 .095 | .422 | 1649 65.0 

“609 .093 .060 | — 907) .418 | .784 53.3 

1609 | — 121 142 1070 | 411 | 787 | 522 

.687 | = 164 .122| — .025 | :514 | .931 55.2 

1651 -168 | — .172| — .061| ‘485 | .836 58.1 

Contribution 

offactor...| 7.441 1:895] 1.000] 1.080]... ecce lernt 


tion with the centroid pattern of Table 8.2 is very close. Asin the preceding 
illustration, the coefficients and calculated communalities of corresponding 
variables differ only slightly. The total calculated communality in the aver- 
oid solution is about 97 per cent of the total centroid communality, illus- 


trating closer agreement than in the preceding case when only thirteen tests 
were employed. 


* An example of such a situation may be found in W. L. Miller, “Тһе Relative Ability 


of the States To Finance Public Education” (Ph.D. dissertation, University of Chicago) 
1940. 


PART III 
DERIVED SOLUTIONS: ORTHOGONAL AND OBLIQUE 


CHAPTER ІХ 
DERIVED PRINCIPAL-FACTOR SOLUTION 


9.1. Introduction 


When any direct solution is obtained, a description of the variables is 
given in the common-factor space thereby determined. The resulting pat- 
tern equations are linear expressions in terms of the common factors for 
theoretical variables which approximate the observed ones. Geometrically, 
any such theoretical variable is the orthogonal projection of the correspond- 
ing observed variable into the common-factor space. The coefficients in а 
pattern equation are the coordinates of a point representing a variable in 
this space. Since the pattern (or reproduced) communalities and correla- 
tions are describable in terms of these coordinates (see 3.8), such values are 
generally different from those used in the original analysis. These discrep- 
ancies have been illustrated in the preceding analyses by the residual cor- 
relations. 

Тһе problem of obtaining a derived principal-factor solution involves the 
transformation of some simply caleulated initial pattern. Тһе initial solu- 
tions which will be employed in this chapter are the centroid, bi-factor, and 
averoid forms, Inasmuch as the principal-factor solution can be obtained 
directly, comparison with the transformed solution can be made in each 
case. It will be convenient to designate the three solutions as follows: 


I = pattern matrix of initial solution, 
P; = pattern matrix of transformed principal-factor solution, based upon the 


matrix I, . 
P = pattern matrix of direct principal-factor solution. 


In obtaining P,, reproduced communalities and correlations based upon 
the values in I are involved, whereas P is obtained directly from the ob- 
Served correlation matrix. Since the reproduced correlation matrix general- 
ly differs from the observed correlation matrix, it is to be expected that the 
Solutions P, and P will be somewhat different. If the solutions I and Р 
fit the original correlations equally well, then the discrepancies between 
P; and P can be regarded as negligible. These points will be illustrated with 


numerical examples. | | 
After presentation of the theoretical basis for transformation to a prin- 


Cipal-factor form in 9.2, several illustrations of the method are given in 
detail. As pointed out in Chapter ҮП, the chief difficulty of the direct anal- 
201 
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ysis for a principal-factor pattern is the large amount of labor involved 
when the number of variables is fairly large. Тһе present method of trans- 
formation is employed to effect a great reduction in the total labor of com- 
putation. The economy of labor becomes more pronounced as the number 
of variables increases. A method for obtaining the roots of a polynomial, 
which arises in the process of transformation, is given in Appendix F. 


9.2. Transformation of a Given Pattern to а Principal-Factor Solution 


Тһе order of procedure for obtaining a derived principal-factor solution 
consists of the determination of a given factor pattern which can be simply 
calculated, followed by a rotation of this coordinate system so that the 
principal axes constitute the new orthogonal reference system. The defini- 
tion of the principal axes given in 7.4 may be taken for this purpose, name- 
ly, those lines which remain fixed under the transformation (7.12). The 
principal axes may also be considered as the orthogonal reference axes on 
each of which the sum of the squares of the projections of the vectors repre- 
senting the variables is stationary. * 

Employing the geometric ideas of Chapter III, any factor, or reference 
variable, may be regarded as a unit vector in the common-factor space of m 
dimensions. In this space the computed variables of the initial pattern are 
represented by vectors whose lengths are less than unity. Let the reference 
vectors of the given solution be designated by Fi, F», . . . , Fm, and let the 
projections upon these axes of the vectors representing the variables be de- 
noted by ал, aj... , dim, (j—1,2,...,n). Denote the principal ref- 
erence vectors by Py, P», . . . , Pm and their direction cosines with respect 
to the axes Fi, P», ..., Pn by 


Qu, №,..., А), (Ха, Хау, Хы) а 
(а Кау , №) 


ы] 


respectively. The problem is to determine the values of these direction co- 


віпев so that the transformation from the known solution to a principal- 
factor solution can be accomplished. 


Тһе correlation between any variable 2, 
may be considered as the sum of the paired products of their respective di- 
rection cosines multiplied by the lengths of their vectors in the common- 
factor space. "Тһе direction cosines of the vector representing z; are (ал/19 
Oi/hj, . . . , Qim/hj), and the length of this vector is h;. The length of the 


and the first, principal factor Рі 


* For general theorems on maxima and minima 
m 


see William F. Osgood, Advanced Cal- 
culus, 1932, Chap. VII. 
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reference vector representing P, is, of course, equal to unity. The desired 
correlation may then be written simply as follows: 


(9.1) т, = ала + ajda +... + jmAmt (/-1,2,...,п). 


This expression will give the coefficients Pa (=7;р,) of the first principal 
factor when the values of the Xs are obtained. Inasmuch as the values 
Рл are the projections of the variable 2; оп the first principal axis, the sum 
of their squares must be maximized. The expression (9.1) for Фа may be 
written in condensed form as follows 


(9.17) "рп = Maa (3-1, 2,...3 7). 


8-1 


Squaring both sides of this expression, and summing over the n variables, 
yields 


(9.2) Ps Dh = Y | Жам) . 


ј=1 
For convenience, the left-hand member of this expression will be desig- 
nated by Aj. 

The sum of the squares of the projections, А1, must be maximized under 
the condition that the sum of the squares of the direction cosines is equal 
to unity. This condition may be written in the form 


(9.3) B= УЎ№-1=0. 
s=1 
In maximizing А; by the method of Lagrange's multipliers,* the function 


(9.4) 2w = А-В 


is employed. In this expression, и is a parameter which will be determined 
1n the following analysis. Consider w as a function of the m variables Xi, 


Aa... › Amy and write the necessary conditions for w to be stationary, 
namely, 

9B, 
9. дю 1(94- )=0 взек Wh De. Й) 
Ms DMa 2\дм “дм (re his sna 


* Ibid. 
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Now, the derivative of A, with respect to any variable X, is given by 


aA, _ п т 
axo 5 алды 


(9.6) 
= 2 anlann + азл +... аһа), 
ізі 
and the derivative of B, is 
aB, _ Е 

(9.7) Эх = 2а (s =1,2,...,m). 
Substituting (9.6) and (9.7) in (9.5), the latter conditions may be written 
explicitly for s = 1,2,..., m, as follows: 

(За) — и) + МаЎалаз +... + Маала = 0, 
(9.8) Хазал + allah — ш) +... + Ami ZA j20 jm =0, 


An ža jma ji + Хаа, +... + Ami(Za?,, — џ) 0, 


where each of the summations extends from j = 1 to j = п. 
A necessary condition for the m equations (9.8) to have solutions (in 
which not all of the values А are zero) is the vanishing of the determinant 


of coefficients. This condition leads to the characteristic equation, which 
may be written in the form 


Za = u) Хауа; ... ада 
Өз) уш=[ A (а-и)... Sanam | a 
Хаад Zaima 0... (Ха, — p) 


Ороп expanding: this determinant, this equation can be written as follows: 


(9.10) FQ) = и" – cy xem 5 (—-1)"e, = 0, 


where the c's are positive numbers. 
can be obtained to any desired degr 
outlined in Appendix Ё. 

Evidently any root р, (s = 1,2,... , т) of (9.10) will make the determi- 
nant in (9.9) vanish and yield a solution for the unknowns in (9.8). The solu- 
tion is labeled (An, Xa, ..., Ami), however, only in the case that the largest 


* Each of the m roots of this equation 
ее of accuracy by methods such as those 


* See Appen. F. 
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Toot ш of (9.10) is employed. When the other roots us, us, . . . , us of (9.10) 
are employed, the sets of direction cosines (xis, Аз, ..., №), ..., (№, 
Aam ++. , №) Of the remaining principal axes are obtained. This follows 
from the preceding form of analysis by a simple substitution of indices. For 
example, in order to obtain the direction cosines of the second principal 
axis, all that is necessary is to replace pj, А, Аз, Bi by ру, Мә, Аз, Bo, re- 
Spectively, and equations (9.1’) to (9.8) follow as before. The new equa- 
tions (9.8) will be formally identical with the old, except that the unknowns 
will be (Ais, Ха)... › №) instead of (An, А, ..., №). The determinant 
of coefficients, nevertheless, will be the same and the characteristic equa- 
tion (9.9), or (9.10), will be unchanged. 

It is clear, then, that the problem of making any sum of squares, A 1, Ag, 

‚ › Аһ, stationary leads to the same characteristic equation. Thus the 
direction cosines of the respective principal-factor axes may, in effect, be 
obtained by employing the roots и, (s = 1, 2, . . . , m) of (9.10) successively 
іп (9.8), changing the second subscript of the X's in each case to agree with 
the index of y. 

Furthermore, it can be shown that the roots of the characteristic equa- 
tion are equal to the respective sums of squares, i.e., и, = А, (s = 1, 2, 

».,m). Hence if a root is very small, indicating that the contribution of 
the corresponding factor is insignificant, the direction cosines of that factor 
axis need not be obtained. 

When all the desired Xs have been determined, it is then possible to 
transform the given solution to the principal-factor form. The first column 
of the transformed pattern is given by equation (9.1) in which the direction 
Совіпев of the first principal axis are now known. The complete transforma- 
tion can be exhibited most conveniently in matrix form, as follows: 


[Qn dig... Gin | Ха Хә... Ат Du Piz +++ Pim 
аз 49%... Gam | № №... № _ | Ba Әз... Pam 
алу Ang... Uam Aut Weiss «x Ман Рт Pn2 +++ Dnm 

It І | 
(9.11) А è T = P 


The pattern matrix A is that of the given solution, T is the matrix of trans- 
formation, and P is the resulting principal pattern matrix. The multiplica- 
tion is performed by the conventional row-by-column procedure, i.e., the 
element рук is the sum of the paired products of the elements in the jth row 
of A by the corresponding elements in the kth column of T. This completes 
the analysis for obtaining a derived principal-factor solution. 
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9.3. Numerical Rotations to the Principal-Factor Form, 
When a principal-factor solution is desired, the most economical pro- 


especially great for a large number of variables. The initial solutions con- 
sidered for this purpose are the centroid, bi-factor, and averoid patterns. 


1. ROTATION FROM CENTROID SOLUTION 
In order to illustrate the meth 


only eight variables involving two factors will be employed. These data 


8, is taken as the initial solution for purposes of rotation. 
This pattern may be exhibited as follows: 


Cr с, 
830 —.396 
818 —.469 
"777 —.470 
c=||-798 -401 
-786 -500 |" 
672 458 
594 444 
647 333 


8 8 


8 
Ха = 44394, Sana = -205, Уа, = 1.5268. 
ns 3-1 ігі 


Тһе characteristic equation may then be written in the form 


z|4394—,) — 5175 
(9.12) uis git Pes [=o 


DERIVED PRINCIPAL-FACTOR SOLUTION 207 


or 
Ди) = 1 — 5.96574 + 6.7285 = 0. 


The two roots of this quadratic equation are readily found to be 
ш = 4.4556, us = 1.5102. 


Employing the larger root, the direction cosines of the first principal- 
factor axis can be obtained by means of equations (9.8), as follows: 


(9.13) | —.01623; — .2175àa = 0, 


-.2175Х4 — 2.929315 = 0. 


In order to obtain unique solutions for the two unknowns Xi, №, it is 
necessary to employ the auxiliary condition (9.3), which in this case is 
Simply 


өм) MM = 1. 


The two homogeneous equations (9.13) have a matrix of coefficients of 
rank one, as is evident from (9.12), and so one of these equations may be 
Used to solve for one unknown in terms of the other. This solution will also 
Satisfy the second equation.* From the first of equations (9.13), the solution 


(9.15) An = —13.43An 
is obtained, which gives an infinite number of values Xi: corresponding to 
arbitrary values of №. To obtain a determinate solution, the condition 
(9.14) is now employed, yielding 

(— 13.43%) + Mi = 1, 


ог 
№ = +.07426. 


Substituting this value in (9.15) gives 
An = +.9973, 


Where the minus sign is associated with the positive value of Аз and vice 
versa. The positive sign will be taken for the larger direction cosine for 
Convenience, i.e., Ап = .9973, An = —.0743. 

The second root, иә = 1.5102, is next employed in order to determine the 
direction cosines of the second principal-factor axis. Substitution of us for u 


*L. E. Dickson, Modern Algebraic Theories, p. 61. 
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in (9.8), and replacing the second subscript on the A's by 2, gives the follow- 
ing pair of simultaneous homogeneous equations: 


2.9292, — 21754; = 0, 
(9.16) | — 2175: + 016135 = 0. 


The solution of these equations, under the restriction 


М» + № =1 , 
is M2 = .0740, № = .9972. 


The direction cosines having been determined, the matrix of transforma- 
tion can be written as follows: 


т -| 9973 .0740 
s —.0743 .9972 || ° 


Then, upon multiplying the centroid pattern matrix C by Tc, the follow- 
ing principal pattern matrix is obtained: 


Р; Р, 
1857 —.333 
851 —.407 
.810 —.411 
ТЖ” 826 —.341 
747 .557 
.636 .506 
.559 .487 
.621 .380 


т Will be noted that there was an ambiguity in choice of algebraic signs 
the 


direction cosines of the principal-factor axes. 
ular set of signs may now be indicated. If the 


adequacy of the solution is not 
signs in а column of à pattern 


The derived principal-factor pattern, the coefficients of which are given 
in the matrix Po, agrees very closely with the solution obtained directly in 
Chapter УП and exhibited in Table 7.4. The coefficients of the first factor 
in the two cases have a maximum discrepancy of only .002, while the coeffi- 
cients of the second factor have a maximum difference of .009. 
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2. ROTATION FROM BI-FACTOR SOLUTION 

An example will be presented now in which a derived principal-factor 
solution is obtained from an initial bi-factor pattern. The illustration is for 
the same set of eight physical variables which were just employed in passing 
from a centroid solution to a principal-pattern. It will be evident then that 
the derived solution from either of these initial patterns is essentially the 
same as the principal-factor solution obtained directly in Chapter VII. 

Тһе bi-factor pattern matrix for the eight physical variables is taken from 
"Table 7.2 and may be written as follows: 


Bo Bi В, 
601 .614 0 
501 .740 0 
.581 .704 0 
B- .598 .652 0 
.694 0 .623 
.611 0 .560 
.562 0 .453 
596 0  473| 
For these data the characteristic equation (9.9) takes the form 
(3.0479 — и) 1.6605 1.3110 
(9.17) f(u) =| 1.6605 (1.8453 — и) 0 -0. 
1.3110 0 (1.1307 — u) 


In the determinant of the characteristic equation arising from a bi-factor 
pattern there will generally be zeros for the elements off the diagonal, except 
for those in the first row and column. This property holds true when the 
Subsets of variables measuring the group factors are distinet. When there is 
some overlap of group factors, then some small values will appear for these 
elements. 

Upon expansion of the determinant in (9.17), the characteristic equation 
may be written 


(9.18) f(u) = u? — 6.02394? + 6.68104 — .0701 = 0. 


This equation may be solved by the methods of Appendix F, yielding the 
three roots 


ш = 4.5631, 
(9.19) u = 1.4502, 
из = .0106. 
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Then, substituting the largest root for р in equations (9.8), the following 
A s 
three equations arise: 


— 1.515201 + 1.6605, + 1.311035 = 0, 
(9.20) 1.66050 — 2.71783 ex. 
1.3110 — 8.432414 = 0, 


where the direction cosines of the first principal axis with respect to the 


three bi-factor axes are denoted by Ха, Au, Аз. The solution for Хо, Mo 
Хь is subject to the restriction 


(9.21) Mi +L + OM, = 1. 


The last two of equations (9.20) may be considered as linearly independent, 


and the variables Ап, № solved іп terms of Ao. The resulting expressions 
may be written 


_ 1.660520. _ EU 
An = 2.7178 = -61097 yo: and № = “3 4304 = 38 0 


By making use of the condition (9.21), the direction cosines of the first prin- 
cipal-faetor axis are found to be 


^o = .8113, Аа = 4957, and Aa = .3099. 
The direction cosines of the second 


principal axis are similarly obtained, the 
values being 


М» = .1678, № = —.7053 , and № = .6888 . 
It will be observed that the third root of the 
-0106, is very small in comparison with the ot 
is equal to the contribution of the third principal factor, this factor will not 
be obtained because of its insignificanee,* 

The two sets of direction cosine 
transformation: 


characteristic equation, из = 
hers. Inasmuch as this value 


в are recorded in the following matrix of 


| 8113 1678 | 
Te = .49%7 —.7053|. 
7.3099 6888 
* Precise methods for determining the si 


Harold Hotelling, “Analysis of a Complex o 
ponents," Journal of Educational P. 


gnificance of principal factors are given by 
of Statistical Variables into Principal Com- 
sychology, 1933, pp. 417-42. 
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By multiplying the bi-factor pattern matrix B by Ts, the derived principal- 
factor pattern matrix is obtained, as follows: 


Pi Р, 
1865 —.317 
.846 —.423 
820 —.399 
б .808 —.360 
‚756 -546 
.669 488 
596 406 
.630 .426 | 


It may be observed that the principal-factor pattern derived from the bi- 
factor solution agrees very closely with the direct solution given in Table 
7.4. The sum of the contributions of the derived principal factors Pi and Р» 
is 6.011, or 99.8 per cent of the total original communalit y used in obtaining 
the direct solution. The slight discrepancies between the derived and direct 
principal-factor patterns are due to the fact that the total communality of 
the latter solution did not agree exactly with the origin 
which the former solution reproduces. 

From the foregoing illustrations, it is apparent that a principal-factor 
form of solution may be derived from either a centroid pattern based upon 
good estimates of communalities or a bi-factor pattern. If the communali- 
ties of two initial solutions are the same, then the derived solutions will 
agree, Such agreement is illustrated by the two derived matrices Pc and P5, 
the minor discrepancies being duc to the slight variations in the communali- 
ties calculated in the initial solutions. In case no initial solution of the pre- 
Ceding type is available and a principal-factor solution is desired, a pre- 
liminary pattern such as the averoid may be employed. А derived principal- 
factor solution may be considered as appropriate provided it is based upon 
апу initial pattern which is statistically adequate. 


al communality, 


3. ROTATION FROM AVEROID SOLUTION 


То illustrate the method of rotation to a principal-factor pattern for а 
fairly large set of variables, the twenty-four psychological tests are em- 
Ployed. After obtaining such a derived solution for these variables, it may 

© compared with the first two principal factors which were obtained di- 
Tectly and are given in Table 7.5. Although both a bi-factor and a centroid 
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solution are available for these data, the present analysis will be based upon 
the averoid solution of Table 8.10. The averoid was selected as the initial 
form of solution in order to indicate the procedure when good estimates of 
the communalities are not available. The total time required to calculate а 
direct principal-factor solution for these data was estimated to be two hun- 
dred hours, while the time required for the computation of the averoid solu- 
tion and the subsequent rotation was about twenty-five hours. The econo- 
my of labor for a set of variables of this size is clearly evident. 

The necessary sums for the characteristic equation (9.9) are calculated 
from the averoid pattern of Table 8.10, producing 


(9.22) flu) = 
(7.4406 — u) —.3136 —.2153 .0360 
—.3136 (1.3950 — м) .0413 —322 | 0 
— 2753 .0413 (1.0987 — ш) -.0017 
10360 —.3229 —.0917 (1.0799 — и) 


This determinant сап be expanded most simply by Laplace's development 


by columns,* using the first two columns as a base, and then the character- 
istic equation becomes 


(9.23) flu) = u* — 11.014245 + 30.52604? — 31.73494 + 11.1727 = 0. 
The complete procedure for the transformation from the averoid pattern 
to the principal-factor form is described in Appendixes F and G. In the 
first of these appendixes а general method for calculating the roots of а 
polynomial of any degree is presented. As an illustration of this method, 
the roots of the characteristic equation of the present example are evaluat- 
ed. There are four roots in this case so that four sets of equations of the 
form (9.8) arise. The solution of such sets of simultaneous equations is рге- 
sented in Appendix G.2, in which the data of this example are used again 
asan illustration. The derived principal-factor pattern is given in Table 9.1. 
Since the axes representing the derived principal factors are obtained by 
rotation of an initial set of axes in а fixed common-factor space, the calcu- 
lated communalities in the two cases are, of course, the same. This agree- 
ment may be seen by comparing the corresponding values of Tables 8.10 
and 9.1. It then follows that the total contributions of these two sets 0 
factors must be equal, which is again evident from these tables. As would 


* See, e.g., L. E. Dickson, First Course in the Theory of Equations, p. 122. 


— 
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be expected, however, the contributions of the individual factors are allo- 
cated differently. 

Тһе coefficients of the first two factors of the above pattern may be com- 
pared with the corresponding values of the direct solution given in Table 


TABLE 9.1 
DERIVED PRINCIPAL-FACTOR PATTERN FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS (BASED UPON THE 
AVEROID PATTERN OF TABLE 8.10) 


Common Factors 

Carcu- 

Test LATED 
|]. маны | жаңа Те | То 

Genera Rigidity patia. emory NALITY 

j Р, Ps Ps Pi № 

.588 - .079 1850 | -.174 .505 

.371 — .047 .278 | —.158 .238 

.421 .109 .821 | —.083 .299 

.480 .087 .208 | —.132 .844 

.682 .309 | — .252 | —.120 .639 

.676 .396 | — .182 .009 .647 

.661 .427 | — .259 | —.092 ‚695 

‚676 `.198 | — .079 | —.131 .518 

.676 .457 | — .151 .043 .690 

.442 — .449 | — .425 | —.118 „592 

‚558 = .812| — .285 .063 .488 

.453 | — .454 | — .089 | —.270 | .492 

.594 — .323 | — .041 | —.271 ‚582 

.422 - .000 | — .071 .407 .349 

.388 — .053 .024 367 - 289 

‚506 — .089 .259 .210 375 

.459 — .170| — .091 .382 .394 

.512 - .819 .151 .226 .438 

.443 = .142 .086 .125 .239 

.615 .116 .161 .062 .421 

‚591 — .230 .082 | —.091 .417 

.608 .085 .174 .054 .410 

.688 .068 .183 | —.049 ‚514 

648 | — .183 | — .179 .007 | .485 

Contribution 
of factor..| 7.467 1.591 1.093 4859: ЕЕЕ 


7.5. The agreement of the individual coefficients is remarkably close in the 
Case of both factors, and the total contributions of the respective factors 
аге practically the same. It would therefore appear that a very close ap- 
Proximation to the principal-factor solution calculated directly from the 
Correlations can be obtained by the preceding rotational method. From the 
foregoing discussion it would appear that, if a principal-factor pattern is 
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desired for a large set of variables, a satisfactory solution can be obtained 
effectively by rotation of an initial averoid pattern. 

Since the first two factors agree with those obtained by the direct anal- 
ysis, they may be named “General” and “Verbal Rigidity,” as in 7.5. The 
third factor has been designated as "Spatial" because the largest positive 
coefficients appear for spatial tests. This factor might also have been named 
*Spatial-Speed" because of the negative coefficients for the speed tests. 
The last factor has been called “Метогу” because of the consistent posi- 
tive weights for the six memory tests. 


CHAPTER X 


ORTHOGONAL TRANSFORMATIONS AND THE 
MULTIPLE-FACTOR SOLUTION 


10.1. Introduction 


In the preceding chapter rotations from simple initial patterns to the 
principal-factor form were discussed. Such rotations were expressed com- 
pletely in analytical form because of the exact mathematical properties of 
the principal-factor solution. Other preferred types of solution may not be 
defined in such precise mathematical terms. The transformation of an ini- 
tial pattern to a preferred type such as the multiple-factor solution, there- 
fore, cannot be developed directly as in Chapter IX but must be constructed 
somewhat subjectively. The guiding principles in formulating such a trans- 
formation are given by the criteria set forth in Table 5.3. 

Before considering the applications of transformations to factor analysis, 
the theoretical development of orthogonal rotations is first presented. In 
10.2 the rotations in the plane are developed and are used as the basis of the 
rotations in three- and higher dimensional spaces in 10.3 and 10.4. Specific 
application to the multiple-factor form of solution is made in 10.5. Other 
types of transformations leading to multiple-factor solutions are available* 
but will not be treated in the present work. Multiple-factor patterns are 
obtained for the eight physical variables, the thirteen psychological tests, 
and the complete set of twenty-four tests. Tn the first and third of these 
illustrations the centroid is used as the initial solution, whereas for the sec- 
ond example the averoid solution is employed. 


10.2. Rotations in a Plane 


When an initial factor pattern involves only two factors, the variables 
may be represented as points in a plane, with the coefficients of the factors 
as the coordinates. Then а transformation to some other form of solution 
implies the representation of these points with respect to the axes denoting 
the new factors. Inasmuch as the origin is assumed to be fixed, such a trans- 
formation is merely a rotation of axes in this common-factor space. The 
transformations considered in this chapter are orthogonal, and hence final 
patterns involve uncorrelated factors. 


* See, e.g., L. L. Thurstone, «А New Rotational Method in Factor Analysis," Psycho- 


metrika, 1938. 
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In derivingan orthogonal transformation in the plane, the required terms 
may be indicated conveniently as follows: 


Fi, Ез = axes representing the original factors, 
Р; = point representing variable z; (0 -1,2,...,), 
ал, aja = coordinates of P; with respect to F^, Fs, i.e., the coefficients in the original 
factor pattern, 
М, М» = axes representing the final factors, 
by, біз = coordinates of P; with respect to Mh, M», i.e., the coefficients in the final 
factor pattern, 
0 = angle of rotation from original to final axes. 


E 


{fom aal 
(by, bya) 


ag 


M, 
Fic. 10.1 


Employing the above notation, any variable 


; may be represented by a point 
Р, referred to either System of refe 5 y 


: rence Fi, P; or Mi, Ma, as shown in 
Figure 10.1. The problem is to express the coordinates (bi, ӛз) in terms 
of the original coordinates (ал, а»). Thisis equivalent to obtaining the final 
factor pattern from the initial one. 


The required transformation is accomplished by making use of the fol- 
lowing property on projections of lines: the sum of the projections upon à 


straight line of the segments of any broken line connecting two points 18 
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equal to the corresponding sum for any other broken line connecting the 
same two points. The points О and P; are joined by two broken lines, name- 
ly, ORP; and OSP;. It follows that the projections of these broken lines 
along any direction are equal: 


(10.1) Proj. OR + Proj. RP; = Proj. OS + Proj. SP; . 


If the direction is taken, first, as the positive axis of M, and, second, as 
the positive axis of М, the resulting expressions are 


(10.2) 


OR cos 0 — RP; sin 6 = 0 + 8Р;, 
ОВ sin 0 + ЕР; cos 0 = OS +0, 


Now, employing the definitions of the coordinates, 


OR = а, ЕР; = а  SP;=bn OS =, 


the result may finally be written in the form 


(10.3) de тшн eem 


bj; = aj sin 6 + aj» cos 0, 


in which the angle 8 is negative. There are n such sets of equations corre- 
Sponding to the number of variables. 

In the above analysis it was convenient to take 0 negative in order to get 
direct expressions for the final coordinates in terms of the original ones. If 
this angle 0 is denoted by —¢, then formulas (10.3) become 


(10.4) bi = ал со8% + аҙ іп ф, 
` bj; = —aj sin ф + aj» cos ф. 


It may be noted that these formulas reduce to (10.3) if the angle ф is nega- 
tive, Hence (10.4) may be taken as the general equations of transformation. 
This transformation may also be expressed in matrix notation, as follows: 


совф — sin ф 
(10.5) lba bel = lan eel | m cos gl: 
ог 
(10.6) M = АТ, 


Where М is the n X 2 matrix of final factor coefficients, A is the n X 2 ma- 
trix of initial factor coefficients, and Т is the 2X 2 matrix of the transforma- 
tion, Formulas (10.4) express the new coordinates (бл, bj2) as functions of 
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the old coordinates (ал, аҙ), involving only one parameter Фф. Thus, when 
а value is assigned to ф, the final factor pattern can be obtained. The man- 
ner in which such an angle of rotation is selected is demonstrated in 10.5. 
To illustrate the analytical properties of orthogonal transformations, the 
foregoing equations may be written in the general form (3.16) as a set of 
linear homogeneous equations: 


ba = №ал + Маа», 
gowy рв = Мап + Аа», 


where the A's are the coefficients expressed in terms of the angle ф. These 
coefficients, by rows, are the direction cosines of M; and М» with respect 


to the original axes F, and Ру and are conveniently indicated in tabular 
form as follows: 


F, Р, 


Mı | муз cos ф Aa = зіп ф 
Mz | M= —sinó А = cos ф 


Inasmuch as the direction cosines satisfy the conditions (analogous to 
eq. [3.15]) 


M+ M, =1, Хайа + АА = 0, and Met =l, 


it is clear that (10.7) represents an orthogonal transformation. 

Besides enabling the analyst to rotate one solution involving two factors 
to another, the above type of transformation forms the basis of the analysis 
for a solution including several factors, It will be shown that a transforma- 
tion in higher dimensional space may be reduced to successive rotations in 
planes. 

10.3. Rotations in Three-Space 


In the case of three factors the transformation from an initial to a final 
pattern may be exhibited as follows: 


ba = ^uaj + Хазар + ^as Қ 
(10.8) bis = Маһ + Ағар + Хаз, 
bjs = 


Mada + Мәзаз + Аза, 
where the notation of 10.2 has bee: 
transformation (10.8) may again 


(10.9) 


n extended to an additional factor. The 
be expressed in matrix form 


М -АТ, 
where, now, Mian X 8, Aisn x 8, апатз$ 3. 
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The elements of T are direction cosines which must satisfy the six inde- 
pendent conditions 
+A + -1, 
М» + Me + Mo = 1, 
Мз + Ms + ARs = 1, 
And + Aado + Aada = 0, 
Mas + ns + Nas = 0, 
Adis + №№ + ХззХзз = 0 . 


Each of these equations is an instance of the following general expression: 


мА + Мм, + NOS = Se 0 (54-1,2,3,6 5), 
which may be written compactly in the form 
3 
(10.10) Ммм =н (= 12,38, 859), 
u=l 


where à,, is the Kronecker à which is equal to unity if s = ¢ and equal to 
zero if s < t. Thus the nine coefficients of (10.8), being subject to six con- 
ditions, afford only three degrees of freedom of rotation in ordinary space. 
Explicit equations for the b’s in terms of the a’s, involving only three in- 
dependent parameters can be obtained.* Since such equations are not em- 
ployed in practical analyses, they are not exhibited here. 

A form of transformation that is not only practical but which can be 
readily generalized to any number of factors will now be discussed. The 
fundamental principle underlying this method is that the result of succes- 
Sive orthogonal transformations is itself an orthogonal transformation. This 
final transformation is said to be the product of the successive rotations. 
Since a planar rotation is the simplest type, a transformation in three-space 
is built up from rotations of essentially this form. Thus a transformation in 
ordinary space may involve the displacement of any two axes about the 
third, being, in effect, a rotation in a plane. Finally, a product of such rota- 
tions may be taken as the complete transformation. 

The rotations can be arranged in a systematic order so that each axis is 
Totated with every other axis only once. The three rotations of pairs of axes 
In ordinary space may be indicated conveniently in the following manner: 


Old Axes Angle of Rotation New Axes 
FF: т Ore 92941 
YiFs ба MiYs 
Ү.Үз 93 M:Ms 


* Virgil Snyder and С. Н. Sisam, Analytic Geometry of Space, 1914, p. 42. 
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It will be noted that the angle of rotation is denoted by 0 with subscripts 
corresponding to the numbers of the axes involved in the rotation. The 
first rotation is made in the plane of F, and F», leaving Ёз unaltered. The 
new axes in this plane are designated by Y, and Ys. Since F; is perpendicu- 
lar to the plane of F, and Fy, it is perpendicular to any line in this plane. In 
particular, Ёз is perpendicular to the new axis Yı. The next rotation is 
made in the plane of У, and Ёз, leaving У; unchanged. The new first axis, 
denoted by M;, may be regarded as final because it is the result of rotations 
with each of the other axes. The last rotation transforms Y; and Y; into 
the final coordinate axes M; and Мз. It will be observed that the Y's are 
merely auxiliary axes and, taken alone, do not form an orthogonal system. 
Both sets of axes Y, У,, P, and М), Y», Үҙ are orthogonal, and either one 
may be taken, in some instances, as the final reference system. The solution 
ordinarily desired, however, is one based upon the complete transformation 
of the original axes F3, Fs, F; to the final My, М», М}. 

Denoting the matrix of transformation of Е, and Fs, leaving Ёз un- 
changed, by 
соз біз —sin бз 0 
Sin O12 соѕ 0. 0 

0 0 1 


Tis = 


the first of the above rotations may be denoted by 


(10.11) Y= APs, 


where A is the initial pattern ma 
coordinates with respect to Yi Y; 
be designated similarly, as follows 


(10.12) 


trix and Y is an intermediate matrix of 
Ёз. The second and third rotations may 


Z = ҮТ 
and 
(10.13) М = ZT», 
where Z is another intermediate matrix, M is the final matrix of coordi- 
nates, and 
cos бз 0 —sin 0; 1 0 0 
Тз- | 0 1 0 $ Т» = |0 соз 054 —sin 003 
Sin бз 0 сов б); 0 sin 053 сов 803} 


The three preceding rotations may be combined into a single transforma- 
tion. Substituting (10.12) for Z in (10.13) gives 


M = YT,;T.; , 
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and substituting (10.11) for Y in the last equation yields 
(10.14) M = ATsTuTa. 
Denoting the product of the three successive rotations by 
(10.15) T = Tuis: 


the expression (10.14) reduces to (10.9). In practice this matrix T cannot 
be obtained directly, but the final pattern is calculated by means of the 
successive rotations. When each of the partial transformations has been 
obtained, however, the complete product matrix T may be recorded and 
used as a check on the coefficients of the final factors. 


10.4. Rotations in Higher Dimensional Space 


The preceding methods can now be generalized to a common-factor space 
of m dimensions. In this space let the initial pattern matrix be denoted by 


А = |la;,ll () =1,2,...„ 9 = LR ... 91m) 


and the final matrix by 
M = |1 (=1,2...,п;з=1,2,...,т). 


Then the transformation from the initial to the final pattern may be ex- 
pressed as follows: 


m 
(10.16) ba = > Мы, 


и=1 
(j51,2,:.::;2;8 = heen 500) 
or in the equivalent matrix form 
(10.17) М = AT, 
Where the matrix of transformation is now 
iu № ... № 
T= An А ... № 
ма hx чуу Hn 
The sets of 7/6, by columns, аге the direction cosines of the final reference 
axes Mi, М.,..., Mm with respect to the original axes Fi, Biss sd 
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These direction cosines are subject to the following set of independent con- 
ditions for orthogonality of the matrix T (see 3.5): 


ы 8,4 
(10.18) Ул. = б, ( і 
ч-1 


Aib 
8 


i 


IA 1 


Since s S t and these indices range from 1 to m, the number of such condi- 


tions is (%) + т or m(m + 1)/2. There is a total of m? parameters іп 
matrix T, and, since these are subject to m(m + 1)/2 restrictions, there 
remain 

m 
(10.19) a сиз 1) e mm ) 


degrees of.freedom of rotation in т-врасе. Тһе number of independent 
parameters given in (10.19) may be associated with the same numberof 
rotations in planes. The planes in which the rotations are made are de- 


termined by all possible pairs of reference axes. "These rotations then in- 


volve j^ angles corresponding to the number of independent parameters. 


As in the case of three variables, the above rotati 
а Systematic manner. For example, 


Scheme may be indicated as follows: 
Old Axes 


ons may be organized in 
when four factors are involved, the 


Angle of Rotation New Axes 
FiF, biz Yi ү, 
YF, біз ДҮ» 
ZF, bu МҮ, 
YoY; Ө» 2.23 
23Ү, 0,4 М.2, 
2:2, 55 М.М, 


10.5. Applications to the Multiple-Factor Solution 


In obtaining a multiple-factor solution, the reference axes are chosen in 
conformity with the discussion іп 5.6. It is first necessary that the initial в0- 
lution shall satisfy the criteria of composition of variables, parsimony of 
factors, and uncorrelated factors. Then an orthogonal rotation of such an 
initial pattern, in its common-factor Space, will preserve these properties. 
The purpose of the transformation is to obtain a final pattern which also 
satisfies the criteria of low complexity, level contributions of factors, and 
hyperplanar fit (i.e., as many zeros as possible in the columns). Ў 

Although any pattern with uncorrelated factors could be used as the ini- 
tial solution from which to rotate toa multiple-factor pattern, in the pres- 
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ent treatment only centroid and averoid initial patterns are considered. The 
procedure is begun by plotting the points representing the variables in the 
plane of the first two initial factors F, and Fe. Since in an initial solution 
of the centroid form the second factor has both positive and negative 
weights, the points will lie in the first and fourth quadrants (see Fig. 10.6). 
The first rotation is then made through an angle 6: such that all variables 
will have positive projections on the new axes Yi, Y». Usually the angle 
біз so selected will be about —45°. Then, by applying equations of the type 
(10.4), the coordinates with respect to Y; and Y; are obtained. The next 
rotation is made in the plane of У, and Рз as indicated іп the scheme exhibit- 
ed in 10.4. The angle біз is obtained again by inspection of the graph. The 
new reference axis Z; should pass near a cluster of points while at the same 
time the other axis Уз also should be near some other points. The variables 
represented by the first cluster of points will have high positive weights for 
the 2, factor, and the variables given by the second set of points will have 
low weights for this factor. Additional rotations may be made according to 
the outline given in 10.4. It will be evident from the following examples 
that the above procedure yields a final solution which agrees with the cri- 
teria of low complexity, level contributions of factors, and hyperplanar fit. 


1. BIGHT PHYSICAL VARIABLES 

The first illustration of a multiple-factor solution is based upon the cen- 
troid pattern of the eight physical variables, given in Table 8.4. The coeffi- 
cients in this pattern are the coordinates, with respect to the two centroid 
axes, of the eight points representing the variables. The plot of these points 
18 given in Figure 10.2, in which it is apparent that the points fall into two 
distinet clusters. If two lines were passed through these clusters of points, 
they would produce excellent geometric fit to the data. Such axes, however, 
аге not orthogonal and therefore not appropriate for the present method. 
Тһе ease of correlated factors will be treated in the next chapter. 

If one axis is passed through a cluster and the other orthogonal to it, the 
Standard of uncorrelated factors is met, but other standards are not well 
satisfied. Thus if an axis М; is passed through the first four points, the other 
axis ЛГ; will be far removed from the second cluster. The coefficients of such 
new factors would have the following properties: 


Variables Coefficients of Mi Coefficients of Mj 
1, 2,8,4 Very high Near zero 
5, 6, 7,8 Fairly high High 


The first four variables would be of complexity one while the last four would 
be of complexity two. Variables 5-8 would not satisfy the criterion of low 
complexity for the present example involving only two factors. 


Его. 10.2 Его. 10.3 


Fra. 10.6 Fie. 107 
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In an attempt to meet the basic standards for a multiple-factor pattern, 
the axes М, and М» аге selected so as to be about equally removed from the 
two clusters of points. By inspection the resulting angle of rotation is taken 
to be 6,; = —42?. Another worker, of course, might select a slightly differ- 
ent angle. The necessary trigonometric functions are 


cos (—42°) = 74341, віһ(-42) = —.6691. 


Substituting these values in equations (10.4), there results 


10.2 
(10:20) bj; = .6691a5 + .7431aj 


үй = .7431ал — .6691a;2, 


which is the transformation from the old coordinates (ал, 6а) to the new 
ones (bj, bj2). Equations (10.20) may also be written in the equivalent form 


1431 .6691 
10.2 4 
(поо) MEg | —.6691 .7431 | ' 


where C is the centroid pattern matrix of Table 8.4 and M is the final pat- 
tern which is presented in Table 10.1. For example, the first entry in Table 


10.1 is given by 
.830(.7431) — .396(—.6691) = .882. 


Тһе pattern of Table 10.1 may be examined now to see how well it con- 
forms to the standards for a multiple-factor solution. Since there are no 


TABLE 10.1 


MULTIPLE-FACTOR PATTERN FOR EIGHT 
PHYSICAL VARIABLES 


^ Commu- 
Variable Mi Ма NALITY 
4 Aj 
1. Height......... .882 .261 .846 
2. Arm врап............ .922 .199 .890 
3. Length of forearm..... .892 .171 .825 
4. Length of lower leg.... .861 .286 .797 
5. МїебрһЬ.............- .250 .897 .867 
6. Bitrochantericdiameter| .193 .790 .661 
7. Chest бігіһ........... .144 .727 .549 
8. Chest width ss] 4258 .680 529 
Contribution of factor....| 3.352 | 2.612 |........ 
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sampling error formulas for this type of solution, the analyst usually must 
set some arbitrary level of insignificance. The present example is based оп 
a large number of observations (N = 305), апа hence the coefficients which 
ordinarily might be judged as insignificant may not be so. In the analysis 
of these data by the bi-factor method, the standard error of a general-factor 
coefficient was shown to have the approximate value .066 (sec p. 141). If 
this standard error were applied to the coefficients in Table 10.1, even the 
smallest value might be judged significant. While the foregoing test is not 
strictly applicable in the present case, it nevertheless throws some doubt on 
the insignificance of the small values. The multiple-factor pattern for these 
data is therefore not a good example of this type of solution. 


2. THIRTEEN PSYCHOLOGICAL TESTS 


The next example is based upon the averoid pattern, given in Table 8.9, 
for the thirteen psychological tests. The transformation to a multiple-factor 
solution in this case is made in accordance with the scheme outlined in 10.3. 
In Figure 10.3 the thirteen points are plotted in the plane of the first two 
averoid axes. The procedure in the present example differs somewhat from 
that employed in the preceding one. The first rotation is made in order to 
accomplish a leveling of the contributions of the first two factors. An angle 
б = 50° is selected by inspection for this purpose. 
points have negative projections on the s 
rotation is immediately follow 


In such a rotation all 
econd axis, designated as Ү). This 
ed by the reflection of the second axis, namely, 


Y: = –ү;, 
80 as to yield positive coordinat 


ciable loadings for both Y, and 
tations. 


es. Although many points will have appre- 
Үз, this сап be adjusted by subsequent ro- 


The equations of transformation (10.4) may be written as follows: 


(10.22) E = .64284, + .7660A, , 
Yi = —.766041 + 64284, , 


where it is understood that for given values of Ay and A 
coordinates with respect to У, and Ү; are obtained. This notation, in which 
the coordinates are not given explicitly, will be found convenient in specify- 


ing the axes when several Successive rotations are involved. The transforma- 
tion (10.22) is followed by 


2 the corresponding 


(10.23) Y; = — Y; = 16604, — .6428A, . 


a 
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Then, writing the resulting matrix of transformation in the form 


т.- | .6428 — .7660| 

12 |7660 —.6428 |” 
and postmultiplying the first two columns of Table 8.9 by Ty», yields the 
first two columns of Table 10.2. The numerical calculations may be checked 
at this stage. The new factors Y;, Y», and the factor Аз are mutually orthog- 
onal, and their total contribution should be the same as that of the original 
System 41, As, As. Thus, 


2.878 + 2.798 + .909 = 4.391 + 1.285 + .909 = 6.585. 
The next rotation is made in the Yi, A;-plane. The plot of points is 


presented in Figure 10.4, in which the coordinates are obtained from the 
first column of Table 10.2 and the third column of Table 8.9. In this trans- 


TABLE 10.2 
INTERMEDIATE COORDINATES 
Test Yi Ys Ys 
1 385 .450 508 
2 246 .269 336 
3 352 .241 496 
4 363 .327 404 
5. 662 .878 065 
6. 725 .293 161 
Ys 718 .298 093 
8. 606 .404 209 
с» 729 .255 107 
LO sane - .048 ‚594 - .890 
11 040 .659 — .156 
12 — .103 .691 — .117 
13 145 .740 137 
Contribution 
offactor...| 2.878 | 2.798 1.083 


formation the first multiple-factor axis is selected. Therefore, it is impor- 
tant that this axis pass near a cluster of points and also be about 90? re- 
moved from a number of other points. То satisfy these requirements, the 
angle 0); = 27° is chosen. In this case one of the axes is reflected again to 
obtain positive coordinates. The final transformation in this plane may be 
Written in matrix form: 


.8829 .4695 
(10.24) |a all = Ys aul oos — 2529] - 


228 FACTOR ANALYSIS 


The resulting values of M; and Уз may be recorded in the appropriate col- 
umns of Tables 10.2 and 10.3. 

The final rotation is made in the Y2, Y;-plane. For this transformation 
the last two multiple-factor axes are selected so аз to pass as closely as pos- 
sible to clusters of points. Thus М» passes near points 10, 11, 12, and 18, 
while М lies close to the points 1, 2, 3, and 4 when the angle of rotation is 


TABLE 10.3 
MULTIPLE-FACTOR PATTERN FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
Test" Verbal Speed Te amA EON 
ј Mi M: M: һ} 
166 216 ‚643 488 
100 116 414 195 
135 028 551 323 
196 143 ‚500 309 
715 323 .208 659 
735 207 .263 652 
764 238 .202 681 
575 290 .350 537 
769 193 .198 668 
153 699 | — .127 528 
128 668 .114 476 
— .054 682 .162 494 
091 628 .415 575 
2.703 2.201 021081. Пасаг 


* The names and brief descriptions of the tests appear in Appen. В.1. 


taken to be 03 = — 23°, 


The transformation to the new axes may then be 
indicated by 


10.25 М, М) = |У, Yi. | -9205 .3907 

Ж IMs ма] = т —.3907 .9205|: 
The coefficients of the factors М, and M 3 are recorded in Table 10.3. The 
numerical check on the total contribution of a f 


actor system, which can be 
made after each rotation, 


may again be employed on the final set of factors. 
Thus the numbers appearing in the last line of Table 10.3 sum to 6.585, 
which is the same as the total contribution of the original averoid factors. 

The multiple-factor pattern of Table 10.3 satisfies the standards listed 
above quite well. The contributions of the factors are relatively level in 
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comparison with other types of preferred solutions. The criteria of low com- 
plexity and good geometric fit also appear to be satisfied. For the present 
sample (N — 145) it is judged that a factor coefficient of two-tenths is 
insignificant. In the present solution at least four entries in each column 
апа one in each row would be considered as essentially zero. The only ex- 
ception occurs for variable 8, which might be considered of complexity two 
ог possibly three if the small value .290 is considered significant. The above 
solution thus affords a good illustration of the multiple-factor type. 

In the naming of the multiple factors, those variables having definitely 
significant weights, say, greater than four-tenths, are considered. The sub- 
groups of tests identifying the multiple factors are the same as those em- 
ployed in naming the group factors in the bi-factor solution of Table 6.13. 
Тһе same names are then assigned to the multiple factors, as indicated in 
Table 10.3. It may be noted that each test is essentially a measure of only 
one of these factors, except Test 13. This variable appears to be a measure 
of both speed and spatial abilities. This description is reasonable inasmuch 
as the test is a measure of speed of perception of simple geometric forms. 


3. TWENTY-FOUR PSYCHOLOGICAL TESTS 


The last example of a multiple-factor solution is based upon the centroid 
pattern of Table 8.2 for the twenty-four psychological tests. The transfor- 
mations follow the scheme previously outlined for four factors. The dia- 
grams from which the angles of rotation are determined are given in Figures 
10.6 to 10.11. Then the successive transformations of coordinates may be 


Summarized as follows: 
7660 .6428 


Y, Р. = lO: Cel вав 7660]: = 40, 
17986 .6018 
Z Yiw-lY: 01| вов тов — = 7975 
9848 —1737|| ^ 
Mi ү.1= 12: С‹Ї-| тзт ом 07 10, 
(10.26 ) 
i . .9962 .0872 " 
а Z,|-lYs Ysl || 0872 |) "7-9 
.8088 .4384 В 
м Zal = 122 Ү‹Ї-| азы | 907 729 
| 8829  .4695 oa, sal A 


Mi Mil| = 12: Z«l*|..4695 .8829 |” 


Его. 10.8 Fig. 10.9 


Fig. 10 11 l 


Fic. 1012 


Fic. 10.13 
Fries. 10.8-10.13 
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Тһе transformed coordinates from the rotations (10.26) are presented in the 
single Table 10.4 for convenience. The axes of the last reference system have 
been designated by primes inasmuch as additional transformations will be 
made. 

Upon inspection of the pattern involving Mj, Mj, Mj, and Mj in Table 
10.4, it is evident from the contribution of the first factor that it probably 
has been given too much emphasis in the successive transformations. The 


TABLE 10.4 
INTERMEDIATE COORDINATES 
Test и | v: Ye | a | Z| 2 м 

1....... 3027 .565|— - 144 
2.. 148} .383|— - 7101 
3 106|  .491|— = -108 
4 145| 466|- = “083 
5 195 151|- 1126 
6 167| 229 1238 
7. 110| .147|— 2095 
8 253| :270|- - 1093 
9 125| .267 .307 
10 646|- :255|— 1022 
11 670|- .057|— 1175 
12 587|- .035|— - - 1096 
13 490  .151|- - — 1089 
14 437| 217 d 
19 392] 238 468 
16 395| .451 409 
17 559| 170 à 
18 587| 1332 1447 
19 386] 203 -347 
20 260| .452]— 1257 
2 474] 1297 2161 
25 281| 1452 1373 
28 319| 485|- - 197 
a 530| 123)— 222 
Contributi 

factor e P |5.416[3.822| 2.487 1.100/4.040|3.450| 2.071| 1.821 

i 


contribution of M! is about twice that of Mj or Mj. In order to obtain a 
more level distribution of the total variance, and a solution with more in- 
appreciable weights for the first two factors and larger weights for the vari- 
ables identifying the last two factors, two additional rotations will be made. 
The twenty-four points are plotted in the planes МІ, Mj and М), Mj in 
Figures 10.12 and 10.13, respectively. By selecting asmall angle of rotation 
in each case, the desired adjustments are made. These supplementary rota- 


tions may be indicated as follows: 


19848 .1737 А 
pr, Mal = dart Mal oes эы, --ш 

(10.27) 9848 .1737 
pr, ма = 1м: |, ш--ш 
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Тһе final multiple-factor pattern is presented in Table 10.5. Тһе checks 
on the calculation of the coefficients in Tables 10.4 and 10.5 after each rota- 
tion are summarized in Table 10.6. Тһе sets of four factors comprising an 
orthogonal system are indicated, and their total contribution is compared 
with that of the original four centroid factors. Since the contributions of the 


TABLE 10.5 
MULTIPLE-FACTOR PATTERN FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS 
Test* Verbal | Speed | Pedue | Memory үсе 
і Mi M: Ms м. һ} 

1, .098 .324 .616 .203 ‚585 

2. .068 -153 .410 .130 .213 

9... .095 .119 ‚582 .181 .823 

Жаз» ‚150 Т .530 -115 ‚348 

Desc .746 .150 .255 .154 .668 

fos .722 .051 .279 .251 ‚665 

Tous .807 .085 .271 m .744 

Boers -535 .258 .379 .140 .516 

Bs. .757 — .039 .287 .805 .750 
TU -277 .664 | — 1101 .139 .573 
11.. .267 -612 | — .045 .286 .530 
12.. .128 -717 .091 .029 .540 
13.. . 239 .634 .312 .021 ‚557 
14.. .231 -188 | — .019 .482 .821 
15.. .106 .138 .078 .500 .286 
16... .053 .218 ‚345 ‚454 ‚375 
1f... .153 .245 | — .027 .616 .464 
18... .006 -386 .199 .522 .461 
10... .123 .223 ‚175 | .392 .249 
20... 807 177 -459 | .292 .422 
21... .172 -460 .931 .245 411 
22.. 281 -17 .400 .399 .430 
23.. .314 .201 .587 .251 .535 
24 .387 .457 .142 .806 .472 
Contribution 

of factor..| 3.430 2.917 2.683 | 2.358 |........ 


* The names and brief descriptions of the tests appear in Appen. B.1. 


respective systems agree with th 
rounding, it may safely be assu. 

Using the same level for jud 
the preceding example, it is cle 


е number 11.383, except for errors due to 
med that the calculations are accurate. Е 
Eing significance of factor coefficients as іп 
ar that the solution of Table 10.5 is a good 
one. There are at least nine insignificant entries in each column, indicating 
good geometric fit. Each column also has a sufficient number of definitely 
significant weights for the naming of the factors. From the descriptions of 
the variables with large coefficients for Mi, М», and M. 4, these factors may ” 
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be appropriately named “verbal,” “speed,” and “memory,” respectively. 
The third factor has appreciable loadings for Tests 1, 2, 3, 4, 20, 22, and 23. 
The first four of these have been called "spatial" tests and involve the de- 
duction of relations among geometric objects. The last three tests involve 
logical and arithmetical relations. The common element of all these tests 
appears to be the deduction of relations, regardless of the specific content 
of the respective tests. The factor Мз might then be named “deduction.” 


TABLE 10.6 
TOTAL CONTRIBUTIONS OF SYSTEMS OF FACTORS 
сс | ү ҮСС | 2үзҮіС | МУЗ | MiZ:Zi¥o | М{М{2,74 | МІМІМІМ | MiM2MsMa 
11.383 11.384 11.385 11.383 11.383 11.383 | 11.382 11.388 


Тһе factor pattern of Table 10.5 for the first thirteen tests may Бе com- 
pared with the pattern of Table 10.3 for the same tests. A remarkably close 
agreement is evident in general. Even in the case of the third factor, which 
is given a different name in the two solutions, there is a close agreement for 
the variables common to the two sets.. The first four tests, which were em- 
ployed in the naming of this factor in Table 10.3, have very similar weights 
in the second solution. Аз pointed out above, however, the high loadings 
on this factor for the three additional tests suggested a somewhat different 
name in the latter solution. This illustrates the principle, stated in 5.8, that 
a factor solution cannot be considered as independent of the set of variables 
employed. Thus the larger battery of tests leads to a different name for the 
third factor and to the inclusion of a memory" factor which is not present 
inthesmallerset. The agreement of the common portions of Tables 10.3 and 
10.5 is evidence of the stability of factor patterns when a particular form of 
solution has been selected. Similar stability was indicated in 6.11 for the bi- 


factor form of solution. 


CHAPTER XI 


OBLIQUE SOLUTION 
11.1. Introduction 


In Chapter V a number of preferred types of solutions involving uncorre- 
lated factors were introduced. Methods for the calculation of such factor 
patterns were developed in subsequent chapters. Now the assumption of 
uncorrelated factors will be discarded, and the procedures leading to oblique 
solutions will be considered. It is clear that a certain simplicity of interpre- 
tation is sacrificed upon relinquishing the standard of orthogonality. This 
disadvantage may be offset, however, if the linear descriptions of the vari- 
ables in terms of correlated factors can be made simpler than in the case of 
uncorrelated ones. Generally this is possible. Hence the preferred type of 
oblique pattern which is proposed is one which approximates the uni-fac- 
tor form. 

The method of analysis of this chapter consists in the rotation of some 
initial orthogonal pattern to an oblique solution.* The communalities of the 
variables, which are determined by the common-factor space of the prelimi- 
nary solution, remain invariant under this transformation. Hence the 
uniqueness for the oblique solution will be the same as that of the original. 
Therefore, the entire development will be made in the common-factor space. 

Тһе geometric setting for the oblique form of solution is presented in 
11.2, where the distinction between pattern and structure is again made. 
This is followed by a detailed outline of procedure illustrated by a simple 
example. Then several additional numerical illustrations are given in 11.4. 
In the final section an illustration is given of the inapplicability of the 
oblique form of solution for variables which lack a clear design. It will be 


shown, however, that a suitable oblique uni-factor pattern can generally be 
achieved when there is evidence of grouping or clustering of the variables 
of a set. 


11.2. Geometric Basis for an Oblique Solution 

In 2.4 the definitions of factor patterns and structures were formulated. 
It was also pointed out that, when the factors are uncorrelated, these Con- 
cepts become identical. In the foregoing chapters, therefore, no distinction 
was necessary, and the term “pattern” was used synonymously with “solu- 
tion." When correlated factors are employed, however, the solution con- 


* For an equivalent method which obviate: 


5 rotation of axes see Karl J. Holzing™ 
“A Simple Method of Factor Analysis," Р, 


sychometrika, IX (December, 1944), 257-62. 
234 
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sists of two essential parts, i.e., the pattern and the structure. Now this 


distinction will be shown geometrically. 
А pattern, in terms of common factors only, may be represented as fol- 
lows: 


2" = bal: + baa +... bis (і-1,2,...,т), 


where b’s are employed to denote coefficients of correlated factors. As point- 
ed out in 3.8, the double prime denotes a variable projected into the com- 
mon-factor space. Since the analysis of this chapter is entirely in the 
common-factor space, the primes will be dropped for simplicity, and the 
foregoing equation written 


(11.1) 2; = bali + djl e+... + bin (1-1,2,..., т). 


The coefficients may be considered as the coordinates of a point 2; with re- 
Spect to the factor axes. This interpretation may be made whether the fac- 
tors are represented by orthogonal or oblique axes. In the latter case the 
theory of 3.6 is employed. 


(b; bj) 
Fra. 11.1 


For the case of two factors these ideas may be illustrated by Figure 1121. 
When the correlation between the factors P and Рз is known, the unit vec- 
tors representing Fı and P; are separated by an angle біз = are COS Tr,r,. 

he oblique reference system is thus determined. The algebraic signs of the 
Coordinates in the “quadrants” are the same as in the conventional orthog- 
onal system. Any variable z; is represented by a vector whose length and 
direction are determined by its coordinates. These coordinates are taken 
to be the coefficients of the respective factors in the analytic expression for 


the variable 
(11.2) 2$ = baFı + БР G= 1,2;: sag T) 
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From the definition of general Cartesian coordinates, given in 3.6, it may 
be noted that the coordinates бул, Ьу» are given by the line segments OP and 
OQ, respectively. For the hypothetical variable Z; the first coordinate is 
positive and greater than unity, while the second coordinate is negative and 
less than unity. 

The length of the vector z; can be determined by means of formula (3.32), 
which in this case may be written as follows: 


2 2 
D*(0z;) = 5,5; cos 0, 
(113) 2 Eam bu 


8-і t=] 
= bjb; соз 614-66, сов 912+ b jab j1 COS 05, + Bj2bj2 COS 022. 


In this formula each of the angles 6; and 622 is equal to zero, and 6; is the 
angle between the reference axes. Hence соз б = cos 65; = 1 and сов ё: = 
тғұғ, The expression (11.3) then reduces to 


(11.4) D*(02;) = bà, + b3, + 2bibisrp p, . 


The right-hand member is the communality ^j of the variable z;, the pro- 
Jection of which is represented in the common-factor space by (11.2). The 


eed of the vector is then equal to the square root of the communality, 
that is, 


(11.5) D(0zj) = h;. 


The geometric interpretation of the correlation of a variable with a fac- 
tor will now be given. Let the angle between the vector z; and the refer- 
ence vector F, be denoted by ф. Also let the projections of the end point 
of the vector 2; upon the F and Р, axes be given by M and N respectively; 


E а іп Figure 11.1. From the right triangle OMz;, it is apparent 
а 


D(0M) 


ades ^1 


This formula reduces to 
(11.6) D(0M) = h; cos ф 


upon making use of (11.5). Formula 
applying (3.51), which for variables z; 


(11.7) 


(11.6) may be simplified further bY 
and Ғ, becomes 


‚ 
Tir, = hihr, cos ф. 
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Since the length, or “communality,” of any factor is unity, this expression 
may be written in the form 
(11.8) Tir, = h; cos ф. 


In this formula, and in all other representations of correlations of variables 
with factors, the prime will be dropped for simplicity. Substituting (11.8) 
into (11.6), the projection of a vector upon a reference axis may finally be 
expressed as follows: 


(11.9) D(0M) = rir, 


In a similar manner it can be shown that the projection, D(ON), of the vec- 
tor 2; on the F, axis is the correlation of the variable with the second factor. 
Of course, the correlation between two factors is also given by the projection 
of either reference vector upon the other. 

By referring to Figure 11.1, the distinction between a coordinate and a 
correlation can clearly be seen. The coordinates may be positive or nega- 
tive and may be greater than one. A correlation coefficient also may be 
Positive or negative but can never exceed unity. It may also be observed 
that the coordinates and correlations approach coincidence as the reference 
vectors approach orthogonality. 

A complete solution involving correlated factors must consist of a pattern 
and a structure. The factor pattern may be exhibited as in equation (11.1) 
Ог, more compactly, in a table giving the coefficients of the factors. The 
structure may be presented in tabular or matrix form. For the preceding 
hypothetical example of two factors and n variables the structure may be 


Tecorded as follows: 


Tg, Tir, 
Tor, Tor, 
Tir, iF, 
Tar, Тағ, 


In addition to the pattern and structure, the oblique solution should also 
include a table of intercorrelations of factors. 


113. Procedure for Obtaining an Oblique Solution 


‚ An oblique solution can be obtained by means of a rotation of some pre- 
liminary orthogonal pattern such as the bi-factor, centroid, or averoid. In 


238 FACTOR ANALYSIS 


this sense, the method of analysis is similar to that employed in the last 
two chapters. Unlike the procedures employed there, however, the angles 
between the reference vectors in the present case are permitted to take on 
values other than 90°, 

А complete theoretical development, employing literal notation through- 
out, would be unnecessarily complex and tend to conceal the actual sim- 
plicity of the procedure. For clarity, then, the development will be applied 
to the example of eight physical variables and may be readily generalized 
to larger sets of variables. Тһе various stages of analysis are presented in 


outline form, including a detailed description of method in each step of the 
procedure. 


1. INITIAL ORTHOGONAL PATTERN 


As pointed out in 2.7 it is advisable in factor analysis to select the vari- 
ables according to a design which is later verified or modified by the factor 
solution. If the resulting pattern reveals distinct subsets of variables, it 
will be especially effective for a clear interpretation of the oblique solution. 
Such a preliminary pattern sometimes may arise without an explicit formu- 


lation of а design. In either case, a pattern furnishing evidence of subsets 
will be regarded as “efficient.” 


Geometrically, the points representing a set of variables which have been 


analyzed into an efficient orthogonal pattern will form distinct clusters. 
For such a configuration new reference vectors can be taken through these 
clusters so as to meet the Standard of linear fit.* In general, of course, the 
new reference system will be oblique. 

If an orthogonal solution for a given set of variables is efficient, then any 
other orthogonal pattern based upon these data is also efficient. The first 
step in obtaining an oblique solution is, then, to select any such initial pat- 
tern. In the present development the centroid solution of the eight physical 


variables, given in Table 8.4, is selected arbitrarily as the initial form. This 
pattern is repeated here for convenience. 


2. SUBSETS OF VARIABLES 


If the initial pattern is an efficient one, the clusters of variables will usual- 
ly be evident by inspection. Another approach is to apply the B-coefficient 
technique} to the original correlations. In the case of large sets of variables 
it is particularly advisable to employ both methods, The latter technique 
decomposes the set of variables into its f 


undamental groups, and then the 
pattern should be inspected for verification. 


* See Standard 5a of Sec. 5.2. 1 See 2.7. 
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For the eight physical variables the B-coefficients were determined in 
4.6 and have the values 


B(1, 2, 3, 4) = 235, В(5, 6, 7, 8) = 179, 


indicating two distinct subgroups. By noting the signs of the coefficients in 
Table 11.1, it is clear that the vectors representing the first four variables 
form a cluster in the fourth quadrant, while those representing the last four 
variables form another cluster in the first quadrant. The above grouping of 
variables is thus verified. 


TABLE 11.1 


INITIAL CENTROID PATTERN OF EIGHT 
PHYSICAL VARIABLES 


COEFFICIENTS OF 
Common Factors 
VARIABLE 

ар ай 
1. Height .830 | — .396 
2, Агпїзрап....... .818 | — .469 
3. Length of forearm 4177 | — .470 
4. Length of lower le; ‚798 — .401 
5. Weight........ EA .786 .500 
6. Bitrochanteric ег .672 .458 
7. Chest girth .594 .444 
8. Chest width.......... .647 .333 
(Сот „чес метан 5.922 — .001 
Contribution of factor....| 4.439 1.526 


In the case of an initial pattern involving only two or three common fac- 
tors, the clusters can be seen directly from the plot of points with respect to 
the originalaxes. The eight variables, whose coordinates are given in Table 
11.1, are plotted in Figure 11.2, where the two distinet clusters are clearly 
evident. Although the figure is very simple for two factors, the geometric 
representation for many factors becomes exceedingly complex. Then the 
methods of B-coefficients and inspection of the coefficients in the pattern 
must be employed. 


3. DIRECTION COSINES OF OBLIQUE REFERENCE AXES 


The next stage in the analysis is to determine the directions of the oblique 
reference vectors. Lines may be drawn, by inspection, from the origin 
through the clusters. The angles which these new axes make with the old 
шау be measured, and the transformation may be determined thereby. 
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This procedure is rather subjective, however, and will be replaced by a more 
objective analytic method. 

Тһе average, or centroid, of each cluster of variables can be determined 
from the coordinates of the corresponding subset of points. These values 


Ета. 11.2 


can be calculated directly from the initial factor pattern. For the illustra- 


d of the cluster 1, 2, 3, 4 is the point whose coordi- 
а 8058, —.4340), while that of the cluster 5, 6, 7, 8 18 
f re 11.2 the new oblique reference axes are drawn 
these points. Although geometric concepts are ran 
T necessary nor feasible to actually plot points an: 
draw the axes for man d i 

y factors. Inst i ints 
матб a stead, the coordinates of these po 


та the initial patter indi nd axes 
through them can be imagined р n, ав indicated above, & 


OBLIQUE SOLUTION 241 


Knowing the coordinates of a point on a line through the origin, the direc- 
tion cosines of that line can be determined. As indicated in 3.5, the direc- 
tion cosines are given by the ratios of the respective coordinates to the dis- 
tance of the point from the origin. Тһе distance of the first point from the 
origin is 

V (.8058)? + (—.4340)? = .9152. 


Then the direction cosines of the line through this point, with respect to 
the C, and C; axes, are given by 


—4340 _ 


.8058 
= 9152 ^ —.4742. 


^u = 79152 = .8805 H Ха = 


These are then the direction cosines of the new axis yı with respect to the 
original axes. In a similar manner, the direction cosines of the үз axis аге 
found to be 

Хо = .8412, Aes = .5406 . 


Now an alternative approach for obtaining these direction cosines will 
be indicated. Instead of employing the-centroid of a cluster, the point сог- 
responding to the composite* of this subset of variables is used. Let the 
composite variables be defined by 
in = 23 + Z2 + 23 + 24, 


чыш Us = 2 + 2% + 2: + 28. 


The standard deviations of these variables may be computed by means of 
(2.40), as follows: 


(111) P = V4 + 2(5.018) = 8.7465, 


в, = V4 + 2(3.820) = 3.4117. 


Then the correlations of the composite variables with the C; and С, factors 
сап be calculated by means of formula (2.43). In this case, the zo of the 
formula is taken as a factor. The correlation of the first composite variable 
with C; is thus given by 


_ 830 + .818 + .777 + .798 _ 
сш 3.7465 


.8603 , 


Where the values in the numerator are taken from Table 11.1. Since the 
Centroid factors are uncorrelated, this value is also the coefficient of C; in 


* See 2.10, 


242 FACTOR ANALYSIS 


the linear expression for the standardized composite variable. In а similar 
manner the remaining correlations, or coefficients, can be obtained and ar- 
ranged in the form of a reduced pattern as follows: 


ш = а. = .8603C, — .4634C2, 
(11.12) ! 
ш = = = .7911C; + .5085C2. 


Such pattern equations have the same properties as those for individual 
variables. 

The coefficients of the reduced pattern may be interpreted as the coordi- 
nates of two points representing ш and us. The direction cosines of the 
lines from the origin through these points can be calculated as before by 
dividing the coordinates by the appropriate distances. These values are 
found to be 


Au = .8804, № = —.4742 
z , 
чыз) | Ха 8412, Ao = .5407. 


The equivalence of the direction cosines of y; and y» with respect to C; and 
Cs obtained by the two methods is proved in Appendix B.10. 

The method of reduced patterns will be employed in the remaining ex- 
amples. Although the arithmetic is somewhat more involved than in the 
method of averages, the reduced pattern furnishes additional interpreta- 
tions of the variables and factors. The correlations of reduced variables 
with factors are also employed for the estimation of factors in the next 


chapter. 
4. INTERCORRELATIONS OF FACTORS 


After the direction cosines of the oblique reference vectors have been ob- 
tained, the correlations between these factors can be determined. Ву еті” 


ploying (3.41) and (3.29), the correlation between ү, and үз in the present 
example is found to be 


Тұл, = 8804(.8412) — .4742(.5407) = .4842.. 


The self-correlations, or variances, of the factors may be calculated in the 
same manner. Thus, 


a, = (8804)! + (—.4742)? = 1.0000, 
o3, = (.8412)? + ( .5407)? = 1.0000. 


а. Алана 
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These calculations serve as a check on the values (11.13) because the sum 
of squares of the direction cosines of any line must be unity. From the cor- 
relation between factors the angle of separation of the reference vectors can 
readily be determined. In the present case this angle is given by 


біз = arc cos .4842 = 6192”, 


5. FACTOR STRUCTURE 
Тһе projections of the vectors representing the variables upon the oblique 
reference axes can now be determined.* As indicated in (11.9), such projec- 
tions are the correlations of the variables with the factors, i.e., the elements 


Fic. 11.3 


of the factor structure. In order to indicate clearly the derivation of the 
formulas for the structure values, Figure 11.3 has been constructed. In this 
figure the oblique reference vectors yı and y; have been taken in the first 
quadrant of the Ci, C» reference system. This has been done merely to 

* It is possible to obtain the coordinates of the points with respect to the oblique ref- 
егепсе system directly by transformation of the original coordinates. Since, in factor 
analysis, both the coordinates and the projections (i.e., the coefficients and the correla- 
tions) are desired, the present approach is suggested as the simplest and the one best 
Adapted to systematic calculations. First the projections are obtained, and then in the next 


Stage of the analysis the coordinates are calculated. 
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clarify the development, the results being the same even if the oblique axes 
fall in other quadrants. The angles from the C; axis to the y; and yz axes 
are denoted by a and B, respectively. 

Any variable z; may be represented in this figure by a point whose co- 
ordinates (aj, aj») with respect to the original centroid axes are the coeffi- 
cients in the initial pattern equation. The variable may also be represented 
by the vector from the origin to the point 2): (ал, аз). The angle from the 
С, axis to this vector is denoted by ф. Then the projection of the vector 2) 
upon the y; axis is given by 


(11.14) D(OM) = р(02,) cos (6 — а). 


As already noted, the projection D(OM) is equivalent to the correlation 
Тууу and the length of the vector is D(Oz;) = h;. Making these substitu- 


tions in (11.14), and expanding the cosine of the difference of two angles, 
this formula becomes 


Tjj, = h; [cos ф cos a + sin ф sin a], 


= (h; сов à) cos a + (h; sin ф) sin а. 


Now, h; cos ф and Л; sin are the projections ад and ау» of the vector 2; 0n 
the C; and С» axes, respectively. Then the formula finally becomes 


(11.15) Tj, = ад соз a + aj; sin а. 
In a similar manner it can be shown that the projection, D(ON), of the 
vector z; on the үҙ axis, that is, the correlation r;,,, is given by 


(11.16) Тр, = ал cos B + aj; sin В. 


The results of (11.15) and (11.16) may be summarized in the following m2" 
trix form: 


Шы?) de rw = das aal. n сова Me = сов) 
№ = sina № = sin 8| ' 


where, in the last matrix, the elements of the first column are the directio? 
cosines of yı with respect to C and C», and those of the second column аге 
the direction cosines of үҙ. Although (11.17) was developed on the basis 0 
Figure 11.3, this expression is true for different positions of the new ахеб y 
and үз, as, for example, that indicated in Figure 11.2. Written in the form 
(11.17), the determination of the elements of a structure can be generalize 
to problems involving more than two factors. 
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For the example the direction cosines are given in (11.13) and may be 
put in.a matrix as follows: 


(11.18) 


.8804 pu 
—.4742 .5407||' 


Then multiplying the centroid pattern matrix of Table 11.1 by this matrix, 
the structure values of Table 11.2 are obtained. The correlations of the 
composite variables with the oblique factors are obtained by multiplying 
the reduced pattern matrix of (11.12) by the matrix (11.18). These values 
are presented in the reduced structure of Table 11.2. 


TABLE 11.2 
OBLIQUE SOLUTION FOR EIGHT 
PHYSICAL VARIABLES 
STRUCTURE PATTERN 
VARIABLE 
Жж re. Lankiness Stockiness 
j J" jn у т 

.919 .484 .894 .051 
.943 .485 .956 —.027 
.907 .399 .932 -.052 
. 893 .454 .879 .029 
.455 ‚932 ‚005 ‚9380 
.374 .813 —.025 .825 
.312 .740 —.060 .769 
.412 .724 .080 .685 

Reduced Reduced 

Structure Pattern 
с анк .977 .473 „977 .000 
Ua. ae o eni .455 .940 .000 .940 


A check on the calculation of the oblique factor structure can be made by 
changing the order of the multiplications and additions in the caleulations 
of the sums of the structure values. For two factors this may be written in 
the form 


(11.19) Aaja + “Уан = Erg, (4-1,2), 


where the range of j is arbitrary but the same for all three sums. To illus- 
trate the application of this check, consider the correlations of the eight 
Variables with yı. Then t = 1 and j ranges from 1 to 8 in (11.19). The check 
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on the first column of Table 11.2 then consists in the agreement of the sum 
of this column with the evaluation of the left-hand member of (11.19). 
© This sum is 
8 
Ўы = 5.215, 
, ігі 
while 
8 8 
Mi > ал + № Уаз = .8804(5.922) — 4742(—.001) = 5.214. 


ізі 1-1 


Since the discrepancy is evidently due only to rounding, the calculations of 
this column of the factor structure are accepted as accurate. Of course, the 
check might have been applied to the entire first column, including the 
composite variables or, again, to the calculations of the correlations of the 
composite variables alone. Similarly, the second column of the factor struc- 
ture is checked. 


6. FACTOR PATTERN 

In order to have а complete solution in terms of correlated factors, the 
linear descriptions of the variables are required as well as their correlations 
with the factors. The pattern coefficients are the coordinates with respect 
to the oblique axes of the points representing the variables and could have 
been obtained directly* from the initial factor pattern. It is more conven- 
ient, however, to caleulate these values after the oblique factor structure 
has been obtained by means of the relationship between the pattern and 
structure which is described in Appendix B.11. 

If the oblique factor pattern for the illustrative example is denoted by 


(11.20) 2j = bay, + Бау, G= 1,2,...,%» 
the problem is to determine the coefficients bj, Әз. For any variable 2) 


multiply (11.20) by yı and Үз, in turn, sum for the М values, and divide by 
N. The resulting equations are 


(11.21) e = ba + Әт,» 
Tin, = Бату, + Әр. 


There is such a pair of simultaneous equations for the determination of the 
two unknowns bj, bj; for each variable 2}. In equations (11.21) the terms 
* See n.*, p. 243. 
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Tiy, and гуу, are known from the factor structure, and the correlation be- 
tween the factors is known from the fourth step in the analysis. The de- | 
terminant of coefficients of the unknowns бл, b;2 is 
1 .4842 

| 4842 1 | 
and remains the same for successive calculations of the b’s. The determina- 
tion of the factor coefficients may be calculated by the method of determi- 
nants, but, especially when many factors are involved, a more efficient pro- 
cedure is desired. Such a procedure is described in Appendix G.3, where the 
present example is used as an illustration. The resulting values of the fac- 
tor coefficients appear in the pattern matrix in Table 11.2. In this table the 
reduced oblique factor pattern is also presented. 


7. CONTRIBUTIONS OF OBLIQUE FACTORS 

After an oblique factor pattern has been obtained, the direct and joint 

contributions of these factors can be determined. The communality of a 
variable гу, as given by (11.1), may be expressed as follows: 


М = bh + OR +... + „+ 2bjbjrr, r, +--+ 
(11.22) 
t 2b; s ibimt ra Fm 


The direct contributions of the factors to z; are given by the first m terms 
in this expression, while the joint contributions of the factors are furnished 
by the remaining terms. The total direct contributions of the factors are 
obtained by summing the direct contributions of the respective factors for 


all the variables as follows 
п п п 
(11.23) Yu. Уњ, ..., Уњ. 
jal j=l j=l 
The total joint contributions of pairs of factors are given by 


(11.24) ree, S baba, ree, Dobbin ЕО » 
ј=1 j=1 


j=1 


These two sets of expressions can be arranged conveniently in a triangular 
matrix in which the direct contributions are put in the diagonal and the 
joint contributions in the remainder of the triangle. 


; 
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In the illustrative example the total direct contributions of the two fac- 
tors are given by 


ІМ 


= .894° + .956? +... + .080? = 3.365, 


> 
5 
S 

Ш 


.051%--(-.027)---... + .685? = 2.613, 


while the total joint contribution of these factors is 


8 
2r,,,, Ба 
ізі 
= 2(.4842)[.894(.051) + .956(—.027) +... + .080(.685)] = —.010. 


The contributions of the respective factors may then be arranged in the form 
of Table 11.3. The grand total of the contributions of a set of oblique fac- 
tors should, of course, be equal to the total communality of the original 


TABLE 11.3 


TOTAL CONTRIBUTIONS 
OF FACTORS 


Grand total =5.968 


solution. In the present example this total 
5.965, which is the sum of the contributions in Table 11.1, The direct con- 


tributions of the factors account for all but a negligible amount of the total | 
common-faetor variance. 


agrees closely with the value 


114. Numerical Illustrations 


Four oblique solutions will now be 
cal tests of 2.8. Two of these will b 
for the sets of thirteen and twenty- 
will be based upon the bi-factor p 
it will be possible to examine th 


presented, employing the psychologi- 
€ based upon initial centroid patterns, 
four variables. The other two solutions 
atterns for the same sets of variables. Then 
€ stability of each pair of oblique solutions 
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It will also be possible to compare the oblique solutions, for a given set of 
variables, based on two different initial patterns. 

Since the same data are employed in all four solutions, the determination 
of the subsets of variables may be indicated now. According to the method 
of B-coefficients of 2.9, the twenty-four psychological tests are grouped into 
the following composites: 


vy = 2% +2 +23 + 24, 

0 = 25 +2 +2, +23 +29, 
(11.25) Vs = 20 + Zn + Zi + 213, 

V4 = 24 + 215 + 216 + 217 + Zis + 219, 

Vs = 229 + Za + 202 + 203 + 2. 
When only the first thirteen variables are employed, these are grouped into 
three subsets which are in agreement with the first three composites of 


(11.25). This grouping is verified in each case by inspection of the initial 
factor pattern. 


1, SOLUTIONS BASED UPON INITIAL CENTROID PATTERNS 


The first illustration is based upon the initial centroid pattern of Table 
8.1. The grouping of the thirteen variables into the three composites, v, v2, 
and оз, is substantiated by the preliminary solution. These composites are 
then employed in the determination of the direction cosines of the oblique 


TABLE 11.4 
REDUCED CENTROID PATTERN FOR THIRTEEN 
PsYCHOLOGICAL TESTS 
Variable о | C: Са D(0uj) 
dibus ce wn .6535 .0735 | —.5012 .8268 
jc еа а Б .8448 .9759 .2330 .9536 
аала -6841| —15697| .1405 | .9013 


reference axes. The standard deviations of the variables vi, v», and оз have 
been computed іп 2.11 and are given by ov, = 2.843, c», = 4.214, and o», = 
3.147. The reduced centroid pattern may then be calculated ud written 
as in Table 11.4. In this table the composite variables, which are in stand- 
ard form, are designated by w’s; and the distances of the points, represent- 
ing these variables, from the origin are given in the last column. The direc- 
tion cosines of the oblique reference axes yı, ys, and уз (which are the lines 
from the origin through the points representing the composite variables) are 
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obtained by dividing the elements (coordinates) in the rows of Table 11.4 
by the respective distances. The resulting values can be recorded in matrix 
form as follows: 


м Мм Na 
Cil .7904 .8859 .7590 
(11.26) C.| .0889 .3942 —.6321 


Сз —.6062 .2443 1559 


In this matrix the direction cosines of the respective oblique axes with re- 
spect to the centroid axes are recorded in columns. 

Тһе correlation between any two oblique factors is obtained simply by 
summing the paired products of corresponding direction cosines of the ref- 
erence axes representing them. In like manner, the variances of the fac- 
tors can be obtained by taking the sum of the squares of the respective 
columns of (11.26). Of course, these variances must be equal to unity, and 
hence they can serve as a check on the preceding calculations. The corre- 
lations among the factors are presented in Table 11.5. If it is desired, the 


TABLE 11.5 
INTERCORRELATIONS OF FACTORS 


angles between these oblique axes can be determined from the correlations. 
In the succeeding analysis, however, the actual values of these angles are 
not required, but only their cosines, i.e., the correlations. 

The oblique factor structure can be determined as indicated by (11.17)- 
Upon multiplying the initial pattern matrix of Table 8.1 by the matrix of 
direction cosines (11.26), the factor structure of Table 11.6 is obtained. 
Similarly, the reduced structure is found upon multiplying the reduced pat- 
tern matrix of Table 11.4 by the matrix of (11.26). The calculation of the 
structure of Table 11.6 can be checked by the application of formula 
(11.19) extended to three factors. 

From the intercorrelations of the factors and the structure, the factor 
pattern can be determined. Employing the method of G.3, the oblique fac- 
tor pattern for the thirteen tests and the reduced pattern are calculated ап 
are presented in Table 11.6 also. The total direct and joint contributions 0 
the three oblique factors are given in Table 11.7. The grand total of the 
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contributions differs from the total original communality only by 0.015, so 
that the entire oblique solution may be said to check. 


TABLE 11.6% 
OBLIQUE SOLUTION FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
STRUCTURE PATTERN 
Test 
Spatial á 
Tin Tir, Tjy, |Relations Verbal Speed 
Yi т з 


‚743 ‚406 430 .131 | - .089 ‚142 
445 264 .204 441 .004 .004 
-604 824 .157 .721 | —.090 | —.142 
-559 886 - 266 .508 .090 | —.003 
451 .807 .429 | —.058 .801 .087 
.489 .807 .339 .037 .809 | —.051 
447 .846 .355 | — -068 .901 | —.030 
.537 A .420 .155 .591 078 
435 .841 311 | —.068 .919 | —.081 
075 81 .712 | —.385 ‚164 .809 
‚302 .357 ‚677 | — .039 -077 .659 
816 .222 .724 .073 | -.177 TIS 
.582 419 .728 .351 | —.061 .594 


Reduced Reduced 
Structure Pattern 


.827 .485 .371 .827 .000 -000 
‚560 ‚954 .440 | —.000 .954 .001 
.405 .416 .901 .000 .001 .900 


аа 
1. . қ 
LEES 


* Based upon centroid solution of Table 8.1. 


TABLE 11.7 
TOTAL CONTRIBUTIONS OF FACTORS 


т тз тз 

1.828 |........|...---... 
- .272 9:894 |... xx 
- .052| — .015 | 2.108 


Grand total —6.981 


The second illustration is based upon the initial centroid pattern of Table 
8.2 for the twenty-four psychological tests. Since this preliminary solution 
involves only four factors, the common-factor space for the succeeding 
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oblique solution consists of four dimensions, at most, according to Theo- 
rem 3.5. Hence only four composite variables are desired, inasmuch as each 
of these variables determines a direction for one of the oblique axes. Of the 
composites (11.25) determined by B-coefficients, the first four agree most 
favorably with the centroid solution of Table 8.2. The points representing 
the variables of each of these subsets fall into distinct *sedecimants"* with 
only two minor exceptions, as may be indicated symbolically by 


u:(t,—,4,—), 
v: (+, 7, =, +), 
v: (+, +, =, —), 
u: (+, +, +, +). 


On the other hand, the points representing the variables of the last subset 
do not fall into a single “sedecimant” but overlap with the preceding sub- 
sets. Thus, from inspection of Table 8.2, variables 20 and 23 might be 
grouped with those of v;, variables 21 and 24 with those of 73, and variable 22 
might be included with those of v4. For simplicity, however, these variables 
аге not regrouped, and the first four composites of (11.25) are accepted. 1% 


TABLE 11.8 
REDUCED CENTROID PATTERN FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS 
Variable с. | Ci | Са | с. | D(0uj) 
CNN 6627 | —.2343 | .3429 | —.2547 | .8225 
MES Lee ауаны .7848 —.3935 | —.3272 .1652 .9514 
Wors э ‚6146 4264 | —.2609 | —.3143 .8965 
L T энә ‚7257 8029 2261 .2930 .8691 


is to be expected that the last five tests will be expressed, in the final oblique 
pattern, in terms of the same factors which аге determined by the composite 
variables most nearly related to them. 

In order to caleulate the reduced centroid pattern, the standard devia- 
tions of the composite variables are required. These were obtained in 2.11, 
and the first three were employed in the previous example. The standard 
deviation of the fourth composite variable is oy, = 3.945. The elements of 
the reduced pattern are obtained by dividing the sums of coefficients of the 
variables comprising a composite by the corresponding standard deviation. 
The reduced pattern is presented in Table 11.8. The direction cosines of 

ЖА "'sedecimant" is one of the sixteen regions into which the four-space is divided by 


the four reference axes. These regions are analogous to the four quadrants in the plane, 
and the eight octants in ordinary space. 
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the oblique reference axes 71, уз, Ys, and y4 with respect to the original frame 
of reference are given in the columns of the following matrix: 


ha X2 Мз Ха 
Ci .8057  .8249  .7525 .8350 
С.| —.2849 —.4136 .4756 .3485 
C3 4169 —.3439 —.2910 .2602 
C,|| —.3097  .1736 —.3506 .3371 


(11.27) 


Тһе correlations among the four oblique factors are obtained as before 
and are presented in Table 11.9. The complete oblique solution, including 
both structure and pattern, for the twenty-four tests is given in Table 11.10. 
Аз a final check on the solution, the contributions of factors are presented 
in Table 11.11. The grand total agrees with the total original communality 
(11.383). The values in the diagonal of this-table indicate the almost level 
contributions of the four oblique factors. 


TABLE 11.9 
INTERCORRELATIONS OF FACTORS 


| Yı | Үз Ya 7“ 
Tes eso ip ds OO ous sca em co woe sn ur ooi 
Nerina иаа 585 | 1:000 |... „е 
Ries tesis нә 458 .463 
LOT 578 .514 


Since the structure and pattern are distinct in an oblique solution, the 
question arises as to how to employ these matrices in the naming of the 
factors. All the elements of the structure are generally different from zero 
because the factors are correlated. This may be seen geometrically from 
the fact that the vectors of the variables usually have appreciable projec- 
tions on all the reference axes. This is true even if a particular vector lies 
directly on one of the reference axes. Thus it appears that the structure is 
not very useful for the naming of factors. On the other hand, when a vec- 
tor lies on one of the reference axes, its end point has zero coordinates with 
respect to the remaining axes. Thus the points representing the variables 
of a subset (lying close to and determining one of the oblique axes) have 
large coordinates with respect to this axis and small coordinates with re- 
Spect to the remaining reference axes. The oblique factor pattern, consist- 
ing of such coordinates, thus approximates the uni-factor form and furnishes 
a basis for clearly identifying the factors. In the case of composite variables 
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TABLE 11.10 
OBLIQUE SOLUTION FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 
STRUCTURE PATTERN 
Test Sadi 
срапа y s 
ELA Tu "0v "jy, Relations | Verbal Speed Memory 
7 | | Vs 7% 7, 
|" 725 .403 .461 -703 | —.068 .091 .035 
2. .102 210 -278 -462 | — 018 | — -009 025 
UM 504 .195 .296 ‚595 .020 | —.081 | — 010 
Аалы 587 . 263 .316 .578 -067 | —.002 | — 051 
LN 449 -419 -398 | —.044 .827 .092 | —.057 
6... 454 836 .440 | —.045 .828 | — 077 .086 
j MR 440 871 -348 | — 051 .931 .040 | — 125 
rm 552 457 414 ‚219 516 164 | — 076 
9. 452 2274 459 | —.076 +903 | — 204 161 
10... .133 .718 .404 | —.316 -081 .TT9 .078 
00. 278 :700 .538 | - 180 066 606 245 
19 345 «715 .360 .138 | — .152 .794 | — 118 
13 536 .689 .392 -350 014 -645 | — 204 
14 203 -328 2534 — .287 176 011 .675 
15. 258 251 .527 | — 068 033 | — 097 .608 
16. -501 ‚316 -567 -307 | — 080 | — ‚069 .471 
175. .236 .379 .657 | —.237 «047 .008 «765 
18” “434 .460 .653 -145 | — 191 -128 | (592 
19 ‚348 .323 .492 .082 -021 .026 .418 
20. .595 .339 .485 .873 .250 | — 050 1171 
oF 538 ‚543 .505 -321 | — 014 .324 .133 
22. .549 | .299 .541 .259 .279 | —.151 -339 
23. 693 .573 .445 .514 489 212 :079 ‚075 
24.. 422 ‚528 ‚609 -539 | — 014 .206 .387 .178 
Reduced Structure Reduced Pattern 
щ.. .823 481 -877 475 .824 000 001 | - 009 
TO esed DO .951 .441 .489 -001 .949 .005 | — 002 
Mass „ьа 411 .415 ‚897 .538 — .001 .003 .899 | — 009 
зз» эи. .502 446 .522 -869 | —.000 | - 002 -002 .869 
* Based upon the centroid solution of Table 8.2, Г 
TABLE 11.11 


TOTAL CONTRIBUTIONS OF FACTORS 


Grand total=11.381 
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the resulting reduced structure and pattern bring out the above relation- 
Ships more clearly inasmuch as the points representing these composites lie 
exactly on the reference axes. 

In the factor patterns of Tables 11.6 and 11.10 the coefficients exceeding 
three-tenths have been printed in bold-face type to facilitate the naming of 
factors. It will be observed that in both of these patterns there are a large 
number of insignificant entries. From the descriptions of the variables, 
given in Appendix B.1, the names of the factors indicated in these tables 
аге justified. 

It may be noted that in the reduced oblique factor patterns all the entries 
аге zero, within errors of rounding, except those in the principal diagonal. 
This follows from the fact that the oblique reference axes are passed through 
the points representing the composite variables. Thus, the reduced uni- 
factor pattern from Table 11.10 тау be written in the form 


щш = 824y1 

и = 949-0 

йз = 89975 

Us = 86974. 


(11.28) 


The reduced oblique pattern for the thirteen variables may be written in a 
Similar manner and agrees with the first three of equations (11.28) to within 
five points in the last decimal place. 

The general stability of an oblique uni-factor solution, upon the reduc- 
tion of the number of variables, can be observed from the patterns of Tables 
11.6 and 11.10. For the thirteen tests which are common to these two pat- 
terns, the corresponding factor weights are quite similar. The discrepancies 
between the insignificant coefficients are of lesser importance than those 
between the large weights. For the latter set the differences are all less 
than one-tenth except for the two values of bs. The two patterns may then 
be considered as essentially the same for the common portions. This agree- 
ment may also be observed in the factor structures and is evidenced most 
clearly in the reduced structures and patterns. Hence the composite vari- 
ables may be taken as the best direct measures of the oblique factors de- 
termined in either solution. 


2. SOLUTIONS BASED UPON INITIAL BI-FACTOR PATTERNS 


In order to make it possible to compare oblique solutions for the same 
variables when based upon different initial patterns, the next two examples 
also will employ the psychological variables but will be based upon the 

i-factor patterns of Tables 6.12 and 6.8. In transforming a bi-factor pat- 
tern to an oblique solution, the number of factors is frequently reduced by 
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one. This is due to the fact that the bi-factor solution, in order to exhibit 
the great simplicity of form, sometimes must involve one more than the 
minimum number of factors possible for an equally adequate solution of 
some other type. The bi-factor pattern can be tested for its rank, say m, 
when it contains m or m 4- 1 factors. Then a reduced bi-factor pattern, 
involving m composite variables, can be calculated. The rank of this re- 
duced pattern is also equal to m. The composite variables may be con- 


TABLE 11.12 
REDUCED BI-FACTOR PATTERNS 


For THIRTEEN P8YCHOLOGICAL Testa For Twenty-Four PevcnorooicAL Tests 


Vari- j| Vari- 
we | Be | Bi | Bs | Bs |DOv)| able 
ш....| 6268] .5332|....../...... 8229| ш.... 
ull] 273221... :6080|..... :9518| ш... . 
us... 2] 15265.:....1...... -7286] .8089| ма... 


TABLE 11.13 
DIRECTION COSINES 
Fon THIRTEEN VARIABLES Fon TWENTY-FOUR VARIABLES 
fa а ^a ^a Ха ^a ^а М 
Бо. 17617 | .7694 | .5857 | Bo..... -7564 | .71 

prova “6480 | O0 0 |в: “6540 | Q^ 6597 тз : © S 

Tos vn à 0 .6388 | 0 nes] O0 1.703. 0 0 0 

Hos es 2 0 0 .8105 | В,..... 0 0 .7515 0 0 

Bi bones 0 0 0 .5143 0 

б» mi 0 0 0 .3673 0 


sidered as m points іп the original bi-factor Space. It can be shown then 
that these m points are contained in a space of т dimensions which also 
contains the origin. Hence a transformation from the bi-factor coordinate 


system to а new oblique reference system of т dimensions can be ac- 
complished. 


The bi-factor patterns of Tables 6.12 and 6.8 can be shown to be (ap- 
proximately) of rank three and five, respectively. The composite variables 
(11.25) are employed, and the reduced patterns of Table 11 12 аге bts i d 
The same notation is used throughout the parallel develo me 5» but the 
reader should have no difficulty distinguishing ee a 


the two examples. D i 
Н Й кыр" 2 . Denoting 
the oblique reference axes by B's, their direction cosines with respect to the 


су 


OBLIQUE SOLUTION 257 


bi-factor reference systems are given in Table 11.13. Then the correlations 
among the factors can be calculated and are presented in Table 11.14. 


TABLE 11.14 
INTERCORRELATIONS OF FACTORS 


Fon THIRTEEN VARIABLES Fon TWENTY-FOUR VARIABLES 


В\ B: Ва 
Жз уз 
а зе 
.451 | 1.000 | 8;..... 
аъ А а к 
7. O -756 | .711 .660 | .775 | 1.000 


TABLE 11.15* 
OBLIQUE SOLUTION FOR THIRTEEN PSYCHOLOGICAL TESTS 


STRUCTURE PATTERN 
Tesr Соммо- 
NALITY 
е * " Spatial Verbal Speed 
j ув, "28, "в, в | Вз Вз A 
usare ‚7483 472 .360 ‚104 ‚049 ‚024 ‚555 
ДЫ нела 450 261 199 .453| —.005| - 001 .199 
3 .589 .284 216 .664 | - 082| —.039 853 
4 558 2354 269 .530 .036 .016 .813 
5 .498 .802 .383 .038 771 .019 .644 
6 ‚460 801 2354 -.012 .809 | —.005 640 
Т еда өза е. 440 857 839 — .084 .924 | — 040 741 
кта БИЕ .703 ‚421 182 NT .090 .527 
9........| 1405 .849 .312 —.124 .950 | — 062 781 
10 .186 .188 .718 —.10| —.115 .814 .539 
11 ‚322 .325 .669 .021 .019 .651 .449 
Is .264 .266 .722 —.047 | -.053 767 .527 
3 .491 .495 ‚725 141 149 .598 .573 
Reduced Structure Reduced Pattern 
MS as e| 823 .482 .307 .823 .000 .000 677 
ма... 558 .952 .429 .001 .951| — .001 .904 
Ur реле) AM .405 .899 .001| —.001 .899 .808 


* Based upon the bi-factor solution of Table 6.12. 


The complete oblique solution for the thirteen tests is given in Table 
11.15. Although based upon a bi-factor pattern involving four factors, there 
are only three oblique factors in this solution. The implication of this is that 
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a solution which fits the data about as well as the bi-factor pattern of Table 
6.12 can be made in terms of one fewer, but correlated, factors. Geometri- 
cally, the thirteen points representing the variables are approximated by a 
three-space in the oblique solution, whereas they are contained in a four- 
space in the bi-factor solution. Hence the communalities as given by the 
former solution are slightly less than those given by the latter. This may 
be noted from the last columns of Tables 11.15 and 6.12. The total contribu- 
tions of the three oblique factors are indicated in Table 11.16, where the 
grand total is also given. This number, of course, falls short of the total 
original bi-factor communality (6.936), because the common-factor space 
is smaller. The difference (0.136) is so small, however, that the greater 
simplicity of having only three factors in the solution is well worth the 
sacrifice. 


` TABLE 11.16 
ToTAL CONTRIBUTIONS OF FACTORS 


1:901 15. мге ках |де 
— 1049 | 3:360 |. ........ 
1000 | — .061 | 2:019 


Grand total 26.800 


The oblique solution for the set of twenty-four psychological tests is pre- 
sented in Table 11.17. Here, again, there is one fewer of the oblique factors 
than bi-factors upon which the solution is based. "Тһе communalities repro- 
duced by the oblique solution are then slightly smaller than those from the 
bi-factor solution. These communalities are not calculated for the individual 
variables, but the total contributions of the factors are given in Table 11.18. 
Тһе grand total of the contributions accounts for 96 per cent of the total 
bi-factor communality. 

As in the case of the two oblique solutions based upon the centroid pat- 
terns, there is excellent agreement for the thirteen variables common to the 
solutions of Tables 11.15 and 11.17. Furthermore, the oblique solutions of 
Tables 11.6 and 11.15 for the set of thirteen variables are in harmony ; and 
this is true also for the two solutions of the set of twenty-four variables. 
The oblique factors identified in any one of these solutions are practically 
identical with those of any of the others. Thus, there is evidence that an 
oblique solution is more or less invariant regardless of the particular form 
of the initial solution. What is important in order that the oblique solution 


shall be “almost unique," however, is that there be clearly defined subsets 
of variables. 


TABLE 11.17* 
OBLIQUE SOLUTION FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 


STRUCTURE PATTERN 
! 
Test | 
"i, | тув, | тв, | тув, | тув, | Spatial | Verbal | Speed | Memory rd 
1 2 3 4 

j B В: Ba 8 Bs 

1 .762 | .419 | .389 | .456 -740|  .001  .001| —.001| .028 
2 456 | .254 | .236 | .277 -434|  .001| —.001| .001 .028 
3 | .617 | .285 | .265 | 1311 .130— .003 —.001| —.000| —.147 
4 558 | .329 | .305 | .359 .484- .001| —.002) — .002 ‚102 
5 440 | .817 | .384 | .451 —.001| .814| —.001| —.002|  .006 
6. .435 | .802 | .379 | .446 —.000| .795, —.002|  .000| .011 
7. 404 | .878 | .352 | .414 —.002| 1.006 —.003| —.002| —.175 
8. .472 | .707 | .412 | .484 .002|  .533| —.000|  .000|  .243 
9 .324 | .840 | .369 | .434 -001| .894 —.001|  .003| —.079 
10. 293 | .276 | .702 | .301 .002(  .001| .793|  .003| —.141 
п. 394 | .370 | .703 | .404 .002|— .000| .636}  .004| .096 
12. .306 | .287 | .749 | .313 .001|— .001| .853| —.001| —.158 
18... 436 | .409 | .709 | .446 .002,— .002| .582) —.000| .192 
14 293 | .276 | .256 | .581 .001]  .000| .002) .701| —.158 
15... 205 | .2: .282 | .517 —.001|— .001 .002 .613| —.124 
16. 375 | .353 | .327 | .506 .003| .002  .001| .457 .136 
It. 819 | .300 | .278 | .694 .001|— .000) .002 | .920| —.294 
18.. -390 | .366 | .340 | .571 .002) — .001|  .001| .429| 181 
19 .334 | .314 | .292 | .445 .000|— .001| —.000| 257;  .243 
20.. 487 | .458 | .425 | .499 .000|  .000| .0001 .000 .644 
21. .488 | .459 | .426 | .500 .001|  .000| .000 | .000 .644 
22 .487 | .458 | .425 | .499 .000|  .000|  .000| 000 .644 
23. .555 | .522 | .484 | .569 —.001|— .001) —.003| —.002| .740 
24. 539 | .506 | .470 | .552 —.000|— .002] — 001) —.003 .718 

Reduced Structure Reduced Pattern 

41. 842 | .453 | .420 | .493 | .637 .842 .000 ‚000 000 ‚000 
ш. 516 | .960 | .450 | .529 | .682 | —.001 .958 -000 ‚002 ‚000 
Was ке, .454 | .427 | .910 | .465 | .600 .000| — .000 .909 .000 .000 
ua. -501 | .471 | .437 | .855 | .663 | —.002 .001| - 001 .851 ‚006 
us. -691 | .650 | .603 | .708 | .914 | —.002|— .002| —.003, — 002 .921 

* Based upon the bi-factor solution of Table 6.8. 

TABLE 11.18 
TOTAL CONTRIBUTIONS OF FACTORS 
Ві Вз Bs Bi Bs 


-401 


2.765 


Grand total =11.246 
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11.5. Illustration of Inapplicability of an Oblique Solution 


Now an illustration will be given in which an oblique solution is not suit- 
able. Such a situation arises when there is no evidence of clearly defined 
clusters either by the method of B-coefficients or by inspection of a pre- 
liminary orthogonal pattern. The example of eight political variables of 7.5 
is employed for this purpose. The principal-factor pattern of Table 7.9 is 
reproduced in Table 11.19 for convenience. 

Although all the variables do not fall into distinct clusters, two subsets 
can be selected by inspection of the principal-factor pattern. It may be ob- 


TABLE 11.19 


INITIAL PRINCIPAL-FACTOR PATTERN AND FINAL OBLIQUE 
SOLUTION FOR EIGHT POLITICAL VARIABLES 


Овічок SOLUTION 
PRINCIPAL- сеж 
Елстов PATTERN 
VARIABLE Structure Pattern 

Pi Ра іт, тут, жу т, 
1 | .69 —.28 74 -.57 76 03 
2 .88 —.48 99 —.69 1.16 22 
3 TT .87 =. 87 --.18 68 | — .24 
4 —.88 —.09 —.79 .87 | — .29 64 
5 .28 .65 03 —.46 | = .85 | —1.18 
6 89 .01 83 —.85 42 | — .52 
7 — .66 —.56 —.42 —.80 53 1.21 
8 — .96 е-, 16 —.84 ‚96 | — .24 77 


served that the groups (1, 2, 3) and (4, 7, 8) are in the fourth and third quad- 
rants, respectively. The eight variables are plotted in Figure 11.4, in which 
the grouping is verified geometrically. It is also evident from this figure 
that the clusters are not well defined in terms of the entire set of variables. 
Passing axes through the composite points determined by the above groups, 
the oblique factor solution is obtained by the procedure of 11.3 and is pre- 
sented in Table 11.19. 

The oblique factor pattern does not approach the uni-factor form of solu- 
tion found in the preceding examples. Instead of being involved in distinct 
groups of variables, the factors are of the bipolar type. The interpretation 
is no simpler than in the case of the original principal-faetor pattern and 
has the added complexity of correlated factors. Furthermore, the total di- 
rect contributions of the two oblique factors are 3.707 and 4.121, respective- 
ly, indicating level variances. Thus, while the essential principal-factor form 


—— 
— 
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is retained by the oblique solution, the important property of decreasing 
contributions of factors has been lost. Thus for the present example an 
oblique solution is evidently not desirable. 


Lol P 
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From the foregoing examples it is evident that the usefulness of the 
oblique form of solution depends upon the design of the set of variables. 
When the variables can be clearly grouped by the method of B-coefficients 
or from any preliminary orthogonal pattern, it is to be expected that an 
oblique solution of the uni-factor form can be obtained. Moreover, such & 
solution is independent of the particular initial pattern and is practically 
unique once distinct clusters of variables are determined. 


PART IV 


ESTIMATION OF FACTORS AND RELATION- 
SHIPS BETWEEN FACTORS 


СНАРТЕВ ХП 
ESTIMATION OF FACTORS 


12.1. Introduction 


There are two basic problems with which factor analysis is concerned. 
Тһе first of these deals with the methods for obtaining the linear resolution 
of a set of variables in terms of hypotheticalfactors. The preceding chapters 
were primarily devoted to the solution of this problem, the results being the 
preferred solutions. The second problem is concerned with the description 
of the factors in terms of the observed variables and is the subject matter of 
the present chapter. 

Since the total number of factors (both common and unique) exceeds the 
number of variables, the value of any partieular factor for a given individual 
cannot be obtained by direct solution but can only be estimated from the 
observed values of the variables. The best prediction, in the least-square 
sense, is that obtained by the ordinary regression method. In 12.2 thelinear 
regression of any factor on the n observed variables is obtained by the usual 
method. Thisis followed by an approximative method, which employs com- 
posite variables, in order to reduce the laborious task of the complete re- 
gression method. А method is presented in 12.4 which is superior to either 
of the preceding because it is more rapid and gives just as accurate results 
as the complete estimation method. This method, in effect, replaces the 
Observed correlations of variables by those reproduced (or computed) from 
the factor pattern. 

Two other methods, which at present do not seem to be as practical as 
the preceding ones, are also given. In 12.5 a regression method is presented 
in which the sum of the squares of the unique factors is minimized. This 
method produces estimates of factors which are usually quite different from 
those given by any of the other methods. The final method for describing 
the factors in terms of the variables involves the mathematical solution of à 
Set of equations rather than the statistical estimation by regression. Hence 
the factors themselves instead of estimates of them are obtained. Unfortu- 
nately, however, this solution is in terms of “ideal” variables (not the ob- 
Served ones) and therefore cannot be employed in a practical situation. 

In order to simplify the development of the following sections, it will be 
convenient to formulate the essential concepts of the previous chapters in 
matrix notation. Let it be assumed that a set of n variables has been ana- 
lyzed in terms of m common factors (Fi, Fs, . . . , Fm) and n unique factors 
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(Ui, Us, ... , Un) as given by (2.16). Such a factor pattern may be written 
in the form of a matrix equation as follows: 


(12.1) Z-MF, 


where the column vectors 


2-і(а 2, asa ДБ) 
(12.2) Fia (Еу FRI. Р. 0, Us: s U,] 


represent the variables* 


1 and factors, respectively, and M is the complete 
pattern matrix, 


ай аз... а, а O tee D 
(123) M = n. Gee ... den 0 8а... 0 
94 а... аһ 0 O а, 


If the matrix of coefficients of the common factors is 
the diagonal matrix of unique-f 
may be written as the composi 


ў represented by А and 
actor coefficients is denoted by U, then M 
te of these matrices, j.e., 


(12.4) M = [A 0. 


TS, but, unless s 
common factors may be correlated, 


іі із 


"++ di @ 0 0 
025 S-|T uj-| m 2.40 a 0 
tm tng tim 0 0 а 

where t; = т.ғ, and a; = Tv 


* Strictly speaking, of course, the variables Should be designated by pri dish 
y primes on the z's. 
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12.2. Complele Estimation Method 


Тһе rectilinear prediction of any common factor F, involves the determi- 
nation of the coefficients in the regression equation 


(12.6) Ё, = Band Biota +... + В.25 +... + Bann 
(s: 21,2, «i sf) s 


where the (n — 1) secondary subscripts have been omitted from the соп- 
ventional notation for regression coefficients. А similar equation can be 
written for any one of the unique factors by employing it in place of F, in 
equation (12.6). By the general theory of multivariate regression, it can be 
shown that the normal equations for the determination of the B's in (12.6) 
may be written as follows: 


| Ba + тев, +... + rum lu, 


(12.7) Taba + Bia t+... + теб = Ё, › 


Ж + Tn2Be2 + s: rd + Ban = ы а 

Thus any factor Р, can be estimated when the correlations (l;a) of the vari- 
ables with this factor and the observed correlations (гу) among the vari- 
ables themselves are known. Of course, if a unique factor U;is being esti- 
mated, then all the correlations of this factor with the variables are Zero, 
except for the correlation with z; which is equal to a;. 

The solution of equations (12.7) can be simply obtained by the determi- 
nantal method (Cramer's rule). To this end, define the matrix D of inter- 
correlations of variables bordered by their correlations with the factor F,, 
as follows: 


1 tt da tne 

üa L dé o Uus 
(12.8) D = |1, та 1 ... т, 

ba Та Те... 1 


Then the regression coefficient of any variable z; in the estimation of F, is 
given by 


(12.9) „= — De б-іж,. 4%, 


Where D,, is the minor of the first element and Dj, is the cofactor of £j, in D. 
It should be noted that D,, is merely the determinant В of observed correla- 
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tions and that the determinants D;, can be expressed in terms of the co- 
factors of the original correlation matrix R. Thus the values (12.9) may 
be written explicitly in the form 


(12.10) Boj = 5 [2.25 + t Ro; +... ЫК], 


where Rs; is the cofactor of rą; in R. 
The regression equation (12.6) for Ғ, may then be written as follows: 
(12.11) F, = 18-2, 


where t, is the column vector fti, t2. . . tns} taken from column s of the 
factor structure S. In particular, if F, is one of the unique factors, say U;, 
this equation becomes 


(12.12) Пле 10: 0 aco Wb osi ШЕ?» 


or, in expanded form, 


(12.13) U; = x [Run + Rojo +... + Rug]. 
If in equation (12.11), s takes оп the values 1, 2,..., m, the results 
can be expressed in a matrix equation as follows: 
(12.14) f= TRZ, 
where f is the column vector {Ё, Fy... Ё„| of common-factor estimates. 


More generally, if the range of s in (12.11) extends over the unique factors 
as well as the common factors, then the matrix equation for the prediction 
of the entire set of factors becomes 


(12.15) F = YRZ, 


where F is the column vector of all the factor estimates. Furthermore, this 
equation represents the estimates for all the individuals if Z is regarded as 
the matrix of all the values of the set of variables. 

An entire set of factors, whether orthogonal or oblique, can be estimated 
by means of equation (12.15). When the factors are uncorrelated, however, 
the elements of M are equal to the corresponding elements of S, i.e., the 


pattern and structure coincide. Then equation (12.15) may be written in 
the form 


(12.16) F = М%-2 (uncorrelated factors) . 
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If the factors are correlated, equation (12.16) obviously does not apply. 
Then the distinction between a pattern and structure must be clearly made. 

It is possible to write a formula for the prediction of correlated factors 
which explicitly employs the pattern matrix M. Let the matrix of correla- 
tions among the common factors be denoted by 


1 ТЕР; ТЕЕ 
ТЕ,Е, 1 TEES 
(12577) ф = 
Тғығ, Тғағу coc 1 


во that the matrix of correlations among all the factors is 


(12.18) b= |9 4! 


The relationship between a factor pattern and a structure is 
(12.19) S-M$, 
which is proved in Appendix B.11. Substituting this expression into equa- 
tion (12.15), the latter becomes 
(12.20) ЕК=ФМ'ЕВ-7, 
upon recalling that Ф is a symmetric matrix. This equation reduces to 
(12.16) when all the factors are uncorrelated. 
Since the unique factors are of minor interest in factor analysis, it is con- 


venient to write a formula for the prediction of the common factors employ- 
Ing the pattern matrix A. Substituting 


(12.21) T - A$, 


Which relation is shown in Appendix B.11, into equation (12.14), this be- 
comes 


(12.22) { = фА'В-!7. 
In сазе the common factors аге uncorrelated, this equation reduces to 


(12.23) f = А'В-—'7 (uncorrelated factors) . 


In the preceding formulas the pattern values are employed instead of the 
Structure values. 
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A measure of the accuracy of estimation of a factor F, by means of equa- 
tion (12.6) is given by the coefficient of multiple correlation, which is desig- 
nated by R,. Several important and useful formulas involving R, will next 
be developed. The normal equations for the regression equation (12.6) may 
be written in the condensed form 


(12.24) Z(F. — Ё.јен = 0 (j21,2,...,9), 


where the summation extends over the N observations of each variable. 
It will be understood, in the following, that Х means “sum for 2 from i = 1 
to i = N,” although the is omitted. Since the set of residuals (F, — F.) 
is orthogonal to each of the т sets of numbers 2;, it is orthogonal to any 
linear combination of these z;. In particular, the set Ё, is such a linear com- 
bination, and hence 


(12.25) У[Р, — ЁЁ, = 0. 
This may be written _ 
=F,F, = ЭЙ}, 
which, upon dividing through by N, reduces to 
(12.26) OT pF, = Cp). 
The coefficient of multiple correlation of F, in terms Of Zi, 2...) Zn is 
defined to be the simple correlation coefficient of F, and F,. The expression 
(12.26) may finally be written in the form 


(12.27) R, = Tp, = TR, 
This formula shows that the standard deviation of the factor estimates is 
equal to the coefficient of multiple correlation. 


A simple formula for the calculation of R, may be given now. Multiply 
both sides of (12.6) by F, and sum for the N individuals, obtaining 


SPF, = Badal’, + В„ә®г»Ё, +... + б„®г„Ё,. 
Upon dividing by N, this equation becomes 
оруға, = Batis + В» +... + Bentre- 
Then, according to (12.27), 
(12.28) R? = Batis + Bass +... + Bantns « 


This is the simplest formula for the computation of the multiple correlation 
coefficient. 
Another formula for the calculation of R, can readily be obtained. Mul- 
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tiply the first of the normal equations (12.7) by Ba, the second by 8.2, etc., 
thus obtaining 


Bali, = Bh F Bubini +... + 8.18.2715 › 
= "^ 2 
(12.29) B,slo, = Baobao + Bos +... + BisBanron › 
Bul = BenBarTni + Bonben t + 0 Pena 
Adding these equations, and employing (12.28), there results 
(12.30) RE = Уб! + 2 У 840a. 
1 


j= j<k=1 


This formula, although not so simple as (12.28) for computing R,, illus- 
trates an important property. Any product term 0,4, in (12.28) measures 
the total (direct and indirect) contribution of the corresponding variable X; 
to R2, or the importance of that variable as a “determiner” of F,. The reso- 
lution of the total contribution of any variable into its direct and indirect 
effect upon F, is indicated іп (12.29). Thus, while 8.10. represents the total 
portion of R? which is due to Xi, the right-hand member of the first of 
equations (12.29) shows that this is composed of the direct contribution 
(82,) of X, and of the indirect contribution (Вит) of X, through its cor- 
relations with each of the other variables X; (k = 2, 3,..., n). It may 
be noted that the indirect or joint contribution of any two variables is 
distributed equally between them. 

While the determinantal method for the caleulation of the regression 
coefficients, given by formula (12.9), has some undisputed theoretical ad- 
vantages, a more economical procedure is desired, especially when dealing 
with a large number of variables. Gauss's method of substitution, which 
is described in Appendix G.1, isemployed for this purpose. The latter method 
produces a routine scheme for the evaluation of the 878 and also furnishes a 
complete check on the arithmetical work. Two illustrations of the complete 
estimation of factors are presented in G.4. These illustrations are for the 
sets of principal factors and oblique factors for the eight physical variables. 

е Тһе values of the regression coefficients in the equations for the predic- 
tion of the oblique factors yı (Lankiness) and yz (Stockiness) are given in 
Table G.5. The equations may be written explicitly as follows: 


qu = 2152; + 89120; + 20325; + .15824: + 04325; 
— .009%; — .0052;; — .00228:, 
(12.31) 
Ja = —.0472; + 1962, — 07222; + 02424; + 61526: 


+ .1962;; + .07327; + 1632. 
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The values of these factors for a particular individual 2 are obtained by sub- 
stituting the appropriate standardized values in the above equations. The 
subscript % has been included in these equations to indicate clearly that 
values are substituted for the variables in order to get particular estimates 


TABLE 12.1 
VALUES OF EIGHT PHYSICAL VARIABLES FOR TWO GIRLS 


Case 1 Case 2 
VARIABLE 
ха zji Xj E 
1. Наш сузо esr wey days eos meme 63.98 in. 0.01 66.34 in. 1.14 
2. Arm span.... В 63.19 іп. -0.42 66.89 іп. 1.06 
3. Length of forearm. . . sa 16.89 in. —0.31 17.99 in. 1.33 
4. Length of lower leg. . E 19.09 in. —0.62 20.71 in. 1.27 
Se МЕЕН. ss ns nir е» әле өз Б 149.25 Ib. 1.98 | 125.5 lb. 0.41 
6. Bitrochanteric diameter. 13.15 in. 1.33 12.44 in. 0.26 
7. Chest бігіһ............ -.| 84.37 in. 1.65 32.52 in. 0.69 
8, Chest Width eias „л eem mmn 10.87 in. 1.42 10.55 in. 0.94 
TABLE 12.2 


MEANS AND STANDARD DEVIATIONS OF EIGHT 
PHYSICAL VARIABLES FOR 305 
FIFTEEN-YEAR-OLD GIRLS 


Variable Mean а Қы 
Deviation 
j Mi А 

EID ——— 63.96 in. 2.09 in. 
2. „Атт Span. eos <% ..| 64.25 in. 2.50 in. 
3. Length of forearm. . . 17.10 in. .67 in. 
4. Length of lower leg. . ...| 19.62 in. . 86 in. 
Б. Wb з» wes excep ew 119.22 lb. 15.19 Ib. 
6. Bitrochanteric diameter....| 12.27 in. .66 in. 
Y, Ghest.girthi.z «c wee sauce ii 31.21 in. 1.91 in. 
5. Chest. Width, ose reisha 9.92 in. .67 in. 


of the factors. In general, however, the secondary subscript is dropped for 
simplicity. 

Тһе prediction of the above factors will now be illustrated for two in- 
dividuals whose measurements for the eight variables are given in Table 
12.1. The original observations X;; are changed to standardized values 
2); by application of formula (2.2). The means and standard deviations 
required for this change are given in Table 12.2. Upon substituting the 
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values z;; into equations (12.31), the resulting estimates become 


Estimates for Case 1: yı = —0.26, ya = 138; 
Estimates for Case 2: yj = 121, ya = 0.53. 


It can be seen from the eight standardized values for each girl in Table 12.1 
that the first is of the stocky type and the second is tall or lanky. While 
the original values reveal these facts, they can be indicated more simply by 
the above factor estimates. Although the estimated factors are not in stand- 
ard form, their standard deviations (.980 and .961) are nearly the same, and 
so the estimated values are approximately comparable. The estimates for 
the first girl indicate clearly that she is almost two standard deviations 
above the mean in the factor *Stockiness" and slightly below the average 
in “Lankiness.” The second girl is a less extreme type—being less lanky 
than the other girl is stocky and also being above average in “Stockiness.”’ 
In the foregoing illustrations standardized values were employed in the 
direct application of equations (12.31). The calculations of the standardized 
values for many variables for a large sample of individuals is laborious. The 
work can be greatly reduced by formally expressing the equations of esti- 
mation in terms of observed values by the use of formula (2.2). Thus, in 
general, an equation of the form (12.6) may be written as follows: 


(12.32) ж, „худ бек. ET 
gi с? On 

where 

(12.33) e = ёам, BE Mat oon +8 М,. 
01 02 On 


It should be noted that an estimated factor is not in standard form. Such 
а variable, however, has a mean of zero and a standard deviation which is 
equal to the coefficient of multiple correlation as shown by (12.27). For data 
which produce a high multiple correlation the estimated factor is almost in 
Standard form. 

The values of the factors estimated by equation (12.6), or (12.32), in- 
clude both positive and negative numbers. If it is desired to eliminate the 
negative values, a transformation can be made to an arbitrary positive 
scale. This can be accomplished by standardizing the values F,; given by 
(12.32) and equating this variable to an arbitrary variable in standard 
form. Thus if the arbitrary variable Y is assigned a mean of 50 and standard 
deviation of 10, the required transformation can be written in the form 


(12.34) у, = 10 p, go, 
F 
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which reduces to 


Is 


(12.35) Y, =, Ё, 4- 50 


БУ 


upon substituting the multiple correlation coefficient for the standard de- 
viation of the estimated factor according to (12.27). Such transformations 
have been found especially useful in comparing factor estimates from a bat- 
tery of psychological tests.* 

An illustration of the proportions of the variance of one of the above esti- 
mated factors due to the eight physical variables will now be given. The 
direct and indirect contributions of these variables on the prediction of the 
factor "Lankiness" are indicated in Table 12.3. Each entry in the table 


TABLE 12.3 


PROPORTIONS OF VARIANCE OF COMPUTED у DUE 
TO THE INDEPENDENT VARIABLES 


Total 
Contribution 
Вз 


.253 
.369 
.184 
.141 
.020 
— .003 
—.002 
—.001 


Variable 1 2 3 4 


a 
е 
EI 
© 


Indirect contri- Р = .961 
bution...... 4277 .216| .143| 216 .018|—.003|— .002| — .001 


proper represents the total indirect contribution (28,;8:,7;,) of variables X; 
and X,. The total indirect contribution of any variable is equal to one-half 
of the sum in the row and column representing that variable and is given 
in the last row of the table. The direct contributions (81;) are given in the 
row preceding the last. The total contribution (81; л) of each variable is 
presented in the last column of the table. Of course, the sum of the direct 
and indirect contributions of each variable must be equal to its total con- 
tribution. Finally, the sum of the entries in the last column (or last two 
rows) is equal to the square of the coefficient of multiple correlation. 


* Karl J. Holzinger and Frances Swineford, А Study in Factor Analysis, 1939. 


ы 
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Although the complete estimation method has been developed and illus- 
trated in the present section, it is not very practical. The general formula 
(12.15), or (12.20), involves the calculation of the inverse of an n X n 
matrix of correlations—a task which is very laborious as n becomes large. 
Тһе present method, nevertheless, forms the basis for the simplified meth- 
ods of the succeeding sections. 


12.3. Approximative Method 

Several approximations for the estimation of common factors have been 
proposed* in order to reduce the matrix whose inverse is required. Such 
approximative methods involve the grouping of certain variables into com- 
posites. The simplest procedure is as follows: Combine the respective sub- 
Sets of variables which best measure the factors and employ formula (12.14) 
in which all the symbols now stand for the corresponding reduced matrices. 
The approximative method is best adapted to the case of many variables 
which fall into a relatively small number of distinct subgroups. In applying 
formula (12.14) to such an example including, say, m subgroups, the m X m 
matrix of intercorrelations among the composite variables is used in the 
place of В. Then the major portion of the calculations is greatly reduced. 
The effect of this procedure is to give all variables of а subgroup equal 
weight, 

If it is desired to give varying weights to some of the individual variables, 
the preceding method can be modified slightly. For example, if the first 
four variables of a set are the best measures of the first common factor, they 
тау be used individually, while all other variables are grouped into com- 
Posites, for the estimation of this factor. The matrix whose inverse is re- 
quired then consists of the intercorrelations of the first four variables and 
the (m — 1) composite variables. When the second, and successive, factors 
аге estimated, the corresponding variables which best measure them are 
retained, while all other variables are grouped into composites. 

To illustrate the approximative method, the data of the eight physical 
Variables are again employed. Grouping the variables as indicated in (11.10) 
апа using the standard deviations of (11.11), the composite variables may 
be written as follows: 


v a+ 22 + 28 + 24 


ш = 


oe, 3.7465 ' 
12.36 
н _ vo _ 2s +Z% + 27 + 28 
Ma ™ Ga 3.4117 : 


* Harry H. Harman, “Systems of Regression Equations for the Estimation of Fac- 
tors," Journal of Educational Psychology, 1938. 
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The correlation between these composites can be calculated by means of 
formula (2.42), yielding 
4 в 
У Тік 
1 > ' _ 5.686 


(12.37) ИКА Gur, 12.782 


= 4448 , 


in which the correlations of the original variables are taken from Table 7.1. 
The estimation of the first oblique factor yı, for example, can be made from 
the regression equation 


(12.38) yı = Buty + Piw. 
The normal equations (12.7) for this case become 


Bu + 44488 = .977, 
(12:80) | 444881 “Б Ві: = .455 " 


where the correlations tn and ta of the composite variables with y; are taken 
from the reduced structure of Table 11.2. The solution of equations (12.39) 
are Bu = .9656, 81 = .0255, so that 


(12.40) Жі = .9656ш + .0255из. 


This equation cannot be used for the estimation of individual values of 
yı unless the values of the composite variables are computed. Rather than 
do this, however, equation (12.40) can be written explicitly in terms of 
the original variables by employing the definitions (12.36). Substituting 
the expressions for ш and иг in (12.40), this equation becomes 


(12.1) т = .258(а + zs + 23 + 24) + .007(25 + 26 + 27 + 23). 


This equation illustrates the fact that all variables of a subgroup are given 
equal weight by the approximative method. For the two girls, considered 
before, the values of yı given by (12.41) are Yu = —.30 and у» = 1.25. 
The slight discrepancies between these values and those previously ob- 
tained are due to the grouping of the variables. 

The preceding illustration is given so that direct comparison, for the same 
data, can be made with the complete estimation method. One would not, 
ordinarily, use the approximative method when only eight variables are in- 
volved. The next illustration is more adapted to the present method. 

Consider the problem of estimating the common factors in the twenty- 
four variable problem of 6.10, the bi-factor pattern for which is given in 
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Table 6.8. The approximative method might be used to advantage for an 
example of this type. For purposes of estimating any of the factors Во, Bi, 
Bz, Bs, the variables measuring By and B; may be grouped to form a single 
composite variable v, as indicated in (2.45), the effect being merely that 
variables 14 to 19 will be equally weighted. Of course, in estimating B, or 
Bs, these variables may be given varying weights. Then to estimate the 
factor В, (Spatial Relations), for example, the following regression equation 
is employed: 


(12.42) B, = B.,21 + B:,22 + В, + 8.24 + Bute + Ваиз + Bu ua + Bus › 


where, for simplicity, a single subscript is placed on each 8 to indicate the 
variable to which it is attached. The B's can be computed by means of а 
routine scheme such as that indicated in Table G.5. The necessary sta- 
tistics are the intercorrelations of the first four variables (in Table 2.2), the 
correlations of these with the four composite variables (in Table 2.5), the 
intercorrelations of the composites (in Table 2.4), and the correlations of 
the variables with the factor (in Table 6.8). The equation of estimation 


becomes 


(12.43) B, = .4682, + .1572 + .4262; + 18224 
4 = Дїйш — .013из = 125u, — 3715. 


This equation can be written explicitly in terms of the twenty-four ob- 
served variables by replacing the standardized composite variables by the 
corresponding sum of original variables divided by the standard deviation 
of the sum, that is, 

(12.44) на i ге; б, 8=1,2,3,4,5), 

va 
where the groups of variables are defined in (2.35) and the standard devia- 
tions are given in (2.46). Then, 


B, = .468г, + .15722 + .4262; + 118224 
— .027(25 + 2 + 2: + 28 + 29) 
(12.45) — 004 (zio + Zu + 2 + 213) 
— .032(zu + zu + Zis + 217 + zu + Z9) 
— .100(2so + 221 + 222 + 223 + 204) - 


Of course, if it is desired, this equation may be written in terms of the ob- 
served variables X;. 
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12.4. Shortened Method 


Recently, a much-shortened method for the appraisal of factors has been 
developed.* This method, which is just as accurate as the complete estima- 
tion method, has the great advantage of replacing the nth order matrix of 
correlations by a matrix of order equal to the number of common factors. 
Since the number of such factors is relatively small compared to the number 
of variables, the labor of computing the reciprocal matrix is greatly reduced. 

The matrix of reproduced correlations, with ones in the diagonal, can be 
expressed as follows:t 


(12.46) R = мм’. 


Substituting (12.4) for M and (12.18) for Ф into this equation, it becomes 


ЕТЕ 


This relation is useful in simplifying formula (12.22) for the estimation of 
the m common factors. Premultiplying both sides of (12.47) by A'U-? there 
arises] 


АЛУ-ЧВ = A'U-«(A$A' + 0°) = (А'0-:Аф + DA’, 


* This method was first presented, for the case of uncorrelated factors, by Ledermann, 
“Оп a Shortened Method of Estimation of Mental Factors by Regression," Psycho- 
metrika, 1939. It was then generalized by Harman, “Оп the Rectilinear Prediction of 
Oblique Factors," Psychometrika, 1941. 


+ The proof of (12.46) can be made by the method indicated in Appen. B.11. The ma- 
trix of observed correlations is given by 
1 


@ R-x 


Zz, 


andif Z — MF is substituted into (i), the matrix of reproduced correlations, with ones in 
the diagonal, is obtained: 


m f ‚м! 1 oe) w 
(ii) R = y MFFM = M (5 FF) M = мем”, 


where the last equality follows from (В.41). In replacing observed by computed correla- 
tions, the tacit assumption is made that the residuals vanish. To avoid additional sym- 
bolism, R is employed for both matrices, but it should be clear when it is computed from 
the observed variables and when it is computed from the factor pattern. 


i Throughout this and the following section it is tacitly assumed that none of the 
uniquenesses vanishes. For an excellent treatment of the contrary case see Guttman, 
“Multiple Rectilinear Prediction and the Resolution into Components," Psychometrika, 
1940, § 17. 
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or, defining the following m X m matrix, 
(12.48) K = АЛ/-“Аф, 
this expression reduces to 
(12.49) (1+ К)А' = AUR. 


Now, premultiplying both members of this equation by (I + К)! and 
postmultiplying by R-!, it becomes 


(12.50) A'R = (I+ К)-А'О-. 
Then, substituting (12.50) for A'R-! in (12.22), there finally results 
(12.51) Е-%(-- K) AUZ. 


Although this formula may appear to be more complex than (12.22), it is 
actually much simpler to apply. Aside from the inverse of the square of the 
diagonal matrix of unique-factor coefficients, which is obtainable without 
any effort, the only matrix whose inverse must be calculated is of order m. 

To show how the common factors can be estimated by the shortened 
method, employing the procedure of Appendix G, it is convenient to write 
formula (12.51) in another form. Premultiply both sides of this equation 
by [6 (I + K)-1]7! to get 


(12.52) (1+ К)ф- = A'U—Z. 


It will be observed that the resulting matrix, оп carrying through the mul- 
tiplications, on each side of (12.52) is of order m X 1. This matrix equation 
thus represents а system of m algebraic equations, obtained by setting the 
corresponding elements equal to each other. The matrices in the right-hand 
member of (12.52) are quite simple, but the expression on the left appears 
to be rather complex. This expression can be simplified, however, by sub- 
stituting the definition of K from (12.48), producing 


(12.53) — (1--Ю%--(1-- A'UA) = ($7 + ATA). 


Finally, the system of m equations for the estimation of the common factors 
may be written in the form of the matrix equation 


(12.54) Lf = A'U-Z, 
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where 
(12.55) B= o> + J and J = AU-A. 


It can easily be shown that the premultiplier of f is a symmetric matrix,* so 
that the system of equations represented by (12.54) can be solved by the 
method outlined in Appendix G.1. 


In case the common factors аге uncorrelated, ф is the identity matrix, 
and formula (12.54) reduces to 


(12.56) 1+ pt = АЛ)-?2 (uncorrelated factors) . 


The recommended procedure in applying formula (12.56) to a numerical 
problem is as follows: Divide each element of the jth row of А by aj, for 
j = 1 to j = n; this gives the matrix U-?A, which occurs as а factor in J 
and the inverse of which also occurs as a factor in the right-hand member 
of (12.56). Then multiply А Бу U-?A, column by column, which yields 
the m X m matrix J. Since J is а symmetric matrix, it is necessary to cal- 
culate only those elements which lie on and above (or below) the principal 
diagonal. Finally, add unity to each diagonal element of J to complete the 
determination of the »i equations represented by (12.56). Then the solu- 
tion for the common factors can be carried out by the method of Appen- 
dix G4. 

The procedure for estimating a set of correlated factors by means of 
(12.54) is quite similar to the preceding. After the matrix J is determined, 
it must be added to $7! instead of the identity matrix. The procedure for 
calculating the reciprocal of a matrix is given in Appendix б.б. When the 
set of equations represented by (12.54) is determined, the solution for the 
oblique common faetors is obtained by the method of G.4. 

То illustrate the shortened method, a set of correlated factors will be 
estimated, and hence the most complex formula can be applied. Again, the 
example of eight physical variables will be used so that comparison with 
the solution by the complete estimation method can be made. In estimat- 
ing the factors yı and y: of the solution given in Table 11.2, it is first neces- 
sary to have the uniquenesses of the eight variables. Since this oblique solu- 
tion is based upon the centroid pattern of Table 8.4, and since the commu- 


* To prove that L is а symmetric matrix it is sufficient to show that the transpose of 
this matrix is equal to the matrix itself, namely, 


І’ = ($ + A'U-*A)' = ($) + (A)'(U-?)'(A')' = p- -A'U7A =L, 
since $ (and hence $-!) is symmetric, as is also the diagonal matrix U-?. 


+ This is the same as the conventional row-by-column multiplication of A’ by U-?A. 


ESTIMATION OF FACTORS 281 


nalities remain invariant, the uniquenesses can be obtained simply by sub- 
tracting the caleulated communalities of Table 8.4 from unity. The matrix 
of coefficients of the common factors and the matrix of uniquenesses follow : 


| .894 051 1154 
| .956 —.027 | 111 
| .932 -.052. 1175 
" | 879 .099/ | 202 
= | os одо, U- 132 
—.025  .825, | .339 
—.060 .769| | .450 
| .080 .685| | 471 


Dividing each element of the jth row of А by the element in the corre- 
Sponding row of U?, the following matrix is obtained: 


5.805 3311 
8.613 —.243 
5.326 —.297 
4.351 .144| 

.038 7.045, i 
—074 2434 
—.133 1.709| 

170 L454| 


Тһе matrix J сап then be caleulated upon the column-by-column multipli- 
cation of A by this matrix, namely, 


(12.57) J = А-А = | 


22.236 miei 
|—.099 10.913 |” 


For the present example the matrix of correlations of the common factors is 


"Y | 1.000 .484 
= | .484 1.000 || ' 


and the inverse of this matrix, calculated in Appendix С.б, is 


(12.58) "T | 1.306 228] | 


—.632 1.305 
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Тһе sum of the two matrices іп (12.57) and (12.58) is given by 


23.542 —.731 
(12.59) І-фі-1- | | . 


--.781 12.218 
Completing the indicated multiplications in (12.54), this equation becomes 


23.5425, —  .731%)| _ || 5.80521 + 8.6132 --...-- .1702; 
| —.731y1 + 12.218y2|| | .331a — .2432 +... + 1.4542, 


П 


or, upon equating the corresponding elements, the following pair of algebraic 
equations arise for the solution of y; and 72: 


ово) [ 28:542171 — 7317, = (5.8052 + 8.6132,--...-- 1702), 
au EC m 12218y, = (331a — .2432, +... + 1.4542) . 


The set of equations (12.60) can be solved by the method of Appendix G, 
as indicated in Table 12.4. Then the equations for the estimation of yı and 


TABLE 12.4 
SOLUTION OF (12.60) By GAUSS'S METHOD OF SUBSTITUTION 


COEFFICIENTS oF 


Тора VARIABLE 
Снкск 
- - 1 2 3 4 5 6 | 7 8 
Forward Solution 
23.542 | — .731 5.805) 8.613] 5.326] 4.351 -038)— .074|— 1133 170 46.907 
- 1. .031 |- .247|— .366|- .220|— .185|— -002 -003 .006|— .007 | — 1.993 
.243|— .297 -144| 7.045| 2.434| 1.709| 1.454 24.795 
:267 .165| 135| — .002|— 4 


.00 A . 00: -005 E 
.024|— .132| -279| 7.046) 2.432| 1.705| 1:459 25.519 
-002 .011|- .023|— .578|- :199|— 140 20 .093 


Back Solution 


Ёз. .... 042 .002|— .011 .023 
і 366 


.578| 1199 
.226| 184 100: 


-140| :120 
.020 


3|— .002 .011 


Үз may be written in full as follows: 


Yı = 2482 + .3662› + .2262; + 1842, + .0202, 
+ .0032, — .0022; + :0112в, 
(12.61) 4 _ 
Үз = .0422, + .0022; — .0112; + .023z, + 57825 


+ 1992, + .1402; + .1202, . 
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"These equations agree very well with (12.31), the minor discrepancies being 
due to the fact that the correlations computed from the factor pattern are 
not exactly equal to the corresponding observed correlations. The coeffi- 
cients of multiple correlation, №, = .978 and R: = .958, indicate that the 
equations (12.61) for the prediction of yı and уз are just as reliable as the 
equations (12.31), for which R, = .980 and R: = .961. For the two girls 
whose measurements on the eight physical variables are given in Table 12.1 
the values of the factors, as estimated by (12.61), are 


yu = —0.28 , Ya = 180 " 
y= 1.22, Yo 056, 


which are practically identical with the values previously obtained by the 
use of (12.31). 


12.5. Regression Method Employing an Alternative Principle 

Instead of the ordinary regression method for estimating factors, another 
method has been proposed by Bartlett.* Whereas in the previous methods 
the sum of the squares of the discrepancies between the true and estimated 
factors over the range of individuals is minimized, in the procedure now to 
be described the sum of the squares of the unique factors over the range of 
variables is minimized. This method is in harmony with Bartlett’s principle 
that unique factors should be introduced only in order to explain discrep- 
ancies between observed values and postulated general or group factors. 
According to Thomson, “Bartlett’s estimates and the regression estimates 
attain different ends, and it is agreed that each method is correct in the 
right place.” t 

Suppose there are just two common factors, so that the factor pattern 
may be written as follows: 


(12.62) 2; = analy + ар”; + ajU; G= haeun Ж), 
The explicit expression for the unique factor of any variable z; is 
(12.63) U; = 1 (2; - anFı — ар”), 
і 
and the sum of the squares of all such factors may be denoted by 
n n 1 
(200 Р, Fa) = SU = D g @ — aaPi ans). 
ј=1 гі 


* M. S. Bartlett, “Тһе Statistical Conception of Mental Factors,” British Journal of 
Psychology, 1937. 
T G. H. Thomson, ‘Methods of Estimating Mental Factors," Nature, 1938, p. 246. 
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Then to minimize the sum of the squares of the unique factors over the 
range of variables, it is necessary that the partial derivatives of the func- 
tion F with respect to F, and F: vanish, i.e., 


ӘР 
ЭР, 


oF 
oF = 22 = ji — алЁу— азЕз)аз = 0, 


1 
= 2X a (z; — алЁу— азЕз)аһ = 0, 
(12.65) 


\ 


where the summations extend from j = 1 to j = n. These equations may 
be put in the form 


а? 
ек MN b 

азал\ p, Qjo| р, 0%, 
(xt59n«(x$n-x3 
where primes have been placed on the F’s to distinguish these estimates of 
the factors from the true factors. These equations involve the two un- 


knowns, Ё{ and Ғ), which may be determined in terms of the a’s and 28, 
ав follows: 


(12.66) 


n 


(12.67) F; = X сий @=1,2), 


j=1 


where the с'в can be evaluated from equations (12.66) by Gauss’s method 
of substitution. 


A set of equations of the type (12.66) can be written, more generally, in 
matrix form, as follows: 


(12.68) Ji = AU-z, 


in which the matrix J is defined by (12.55). Formula (12.68) can be applied 
to problems involving any number of common factors if it is desired to esti- 
mate them according to the principle of minimizing the unique factors. Al- 
though formula (12.68) is not a special case of formula (12.54), it may be 
noted that, if the term ф-1 is dropped from the matrix L in (12.54), that 
formula becomes identical with (12.68). The computations required in the 
application of formula (12.68) are the same as that described for the short- 
ened method, except that nothing is added to the matrix J. 
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The estimates of the common factors by means of formula (12.68) are 
then an alternative solution to any of the regression estimates of the pre- 
ceding sections. Bartlett pointed out that the principle of estimation adopt- 
ed in this section does not completely agree with the solution that has usual- 
ly been employed, although the difference docs not affect the relative 
weights assigned to the variables in estimating a single general factor. When 
several common factors are involved, however, the discrepancy between 
equations (12.54) and (12.68) is even more serious. Bartlett states: “Опе 
point of view appears to have been to consider all the persons with different 
possible factorial make-ups that would give rise to the observed test scores 
of a particular person, whereas I have regarded the test scores as a sample 
of all the possible scores that might have arisen for that person according to 
the different values of specific [unique] factors he may happen to have."* 

The present method will be applied to the estimation of the two oblique 
factors, yı and уз, which have already been predicted by means of formulas 
(12.14) and (12.54). The matrix J for this problem is given in (12.57), and 
the transpose of the matrix (A'U-?) is given immediately above it. Then 
the set of equations represented by (12.68) can be solved by the method of 
Appendix G, producing 


ү! = 2612 + 3872; + .24023 + .1962: + .0052; 


— .0022, — .0052; + .008z; , 
(12.69) 
4 0332 — .019z; — .0252; + .015z, + 6462; 


+ 223% + .1572; + .1832. 


ә 
S 
І 


For the two girls previously considered the factor estimates by (12.69) are 
аз follows: 


yn = —0.85, ya = 2.03; 
Yo = 1.28, ү» = 0.56. 


The reliability of prediction by means of equations (12.69) can be judged 
by the appropriate standard error or multiple correlation coefficient. It 
should be noted, however, that formula (12.28) does not yield the required 
multiple correlation for the present method. If formula (12.28) is applied, 
the resulting “multiple correlation" will be found to be equal to unity. This 
follows from the fact that the common factors are estimated under the con- 
dition that the unique factors are minimized, and the uniqueness is the 
standard error of estimate in a pattern equation. 


* Op. cit. 
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For the method discussed in this section the coefficient of multiple corre- 
lation is given byt 


Ju 
(12.70) a Mad 


where J,, is the minor of the element in row and column s of J. In general, 
it can be shownf that the multiple correlation coefficient can be computed 
from the formula 
L 
Om EM 
(12.71) #-1- 7, 


where L is defined in (12.55) апа L.. is the minor of the element іп row and 
column s of L. 
The multiple correlations for the estimates (12.69) as given by (12.70) are 


В, = Vi — 10.913/242.652 = .977 , 
В, = V1 — 22.236/242.652 = .953. 


These values are just about equal to those obtained for the estimates 
(12.31) and (12.61). It may then be concluded that, statistically, the dif- 
ferent methods for estimating factors are equally good. The choice of the 
method of this section instead of one of the other (equivalent) methods must 
be made on the basis of the principle of prediction which is involved. 


12.6. Factors Expressed in Terms of Ideal Variables 


Before leaving the subject of estimation of factors, one more approach 
will be presented. The present method is designed only for the description 
of the common factors—the expressions for these factors being in terms of 
hypothetical rather than the observed variables, and so are not immediately 
applicable in practice. 


Тһе common-factor portion of a pattern is required and may be written 
in matrix form as follows: 


(12.72) Z* = M, 
where f is the column vector (Fi Рз 


Hn 


н 2... Fm} of common factors, and Z* is 
the column Vector |217 zj' . 


2. 247 of the statistical variables projected into 
1 See Bartlett, ibid. 


i For the orthogonal case see Guttman, op. cit., $ 10, and Dwyer, “Тһе Evaluati 
of Multiple and Partial Correlation Coefficients from the Factorial Matrix poen 
metrika, 1940, p. 229. " 


ESTIMATION OF FACTORS 287 


the common-factor space. To solve for the factors in terms of the hypo- 
thetical variables (2/”), proceed as follows: Premultiply both sides of (12.72) 
by A', obtaining 


(12.73) A'Z* = ААА. 
Let 
(12.74) А = А'А, 


which is readily seen to be a symmetric matrix of order m. Finally, pre- 
multiply both sides of (12.73) by A~ to obtain the result 


(12.75) f = А-А?%. 


This matrix equation gives the linear descriptions of the factors in terms 
of the variables z// which are involved in the matrix Z*. 

To illustrate the present method, formula (12.75) will be applied to the 
example of eight physical variables. The matrix A of coefficients of the com- 
mon factors y; and y; is given on page 281. The matrix A, calculated ac- 
cording to its definition (12.74), and the inverse of this matrix follow: 


a =|| 3365 —011 "INE s 
5 |012 2.613 ||’ 001 .383 [1° 


Тһе inverse matrix may be computed by ће method outlined in Appen- 
dix б.б, or, for the simple case of a second-order matrix, directly according 
to the definition of an inverse in Appendix A. The only caleulation remain- 
ing is the multiplication of A^ by A'. Then the expressions for the factors 


can be put in the form 


qi = .2662/' + 28424! + 27724 + .26124' + 0022) 
— 0072 — .0172/' + .02424", 


(12.76) 
ү» = 0202)! — 00924’ — 0192! + .012zj' + .3562; 


+ .3162// + .294z;' + .26225’ . 


Equations (12.76) give the descriptions of the two oblique factors (not 
their estimates) in terms of the **common-factor portions” of the original 
variables, The values of the variables z/’ are not known, and hence (12.76) 
cannot be applied directly to obtain the values of the factors for the indi- 
viduals. An approximation can be made, however—replacing each zj’ by 2;- 
When this is done, the estimates of the factors may be denoted by double 
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primes. Thus, the estimates of y; and y; by (12.76), employing the values 
of the eight physical variables given in Table 12.1, become 


Estimates for Case 1: үү = —.036, — 441 = 1.98; 
Estimates for Case 2: yj; — 131, Үш = 0.68. 


Тһе method of the present section can be applied in another situation 
with less misgiving than that involved in the foregoing example. When a 
factor analysis is made under the assumed composition of variables into 


common factors only,f then the investigator is justified in replacing the Z* 
matrix by the Z matrix throughout this section. 


T This assumption implies the following linear expression for the approximation of any 
variable z; instead of the expression (2.4): 
zi = ааЁ + аР +... + аР (-1,2,...,п), 


where m < n and the prime is used to distinguish the theoretical from the observed vari- 


able. Under this assumption the analysis is carried to the point where there are no more 
statistically significant (common) factors. 


CHAPTER XIII 


RELATIONSHIPS BETWEEN DIFFERENT 
FACTOR SOLUTIONS 


13.1. Introduction 

The present chapter is designed to show the essential unity of factor 
analysis. Although the preferred solutions which yield the linear expres- 
sions of the variables in terms of factors have certain distinct character- 
istics, they can be transformed from one into another. The problem of relat- 
ing two sets of factors, obtained by different methods of analysis, was first 
considered by the authors in 1937,* but the present treatment is somewhat 
more general. 

In 13.2 the general procedure for transforming one solution into another 
is presented. This is followed in 13.3 by a numerical illustration, employing 
the bi-factor and oblique solutions of the set of thirteen psychological vari- 
ables. Finally, in 13.4 the complete plan for obtaining a factor solution is 
outlined and should serve as a summary of the methods of factor analysis. 


13.2. Relationships among Factors 

In mathematics the reference system plays a very minor role; the particu- 
lar configuration of points is of prime importance, and the coordinate system 
is of much lesser significance. Thus, if it is desired to describe an ellipse, i.e., 
get an algebraic equation for the ellipse, it is quite irrelevant whether rec- 
tangular Cartesian coordinates, nonrectangular Cartesian coordinates, or 
polar coordinates are employed. Furthermore, the particular orientation 
of axes is immaterial. With each change of the coordinate system, of course, 
the equation of the ellipse will generally change, but the fact remains that 
the equation in each case describes the ellipse with respect to the given ref- 
erence system. 

The object of factor analysis, on the other hand, is the selection of an ap- 
Propriate frame of reference, the configuration of points representing the 
variables being of lesser significance. Then the indeterminacy of the factor 
Problem is obvious. In selecting a particular reference system, the unit vec- 
tors along the coordinate axes represent the factors, and, since the reference 
System can be rotated about its origin in an infinitude of ways in the com- 
™Mon-factor space, there arises an infinite number of factor systems for a 
given body of data. 

* Holzinger and Harman, "Relationships between Factors Obtained from Certain 
Analyses," Journal of Educational Psychology, 1937, pp. 321—416. 
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Тһе problem of relating two factorial solutions involves the determina- 
tion of the relationships among the coordinates of the two systems. This 
may be accomplished by finding a matrix of transformation which carries 
the coordinates of one factor pattern into another. Thus, if the first factor 
pattern is denoted by A and the second factor pattern by B, then the prob- 
lem is to find a matrix T such that 


(13.1) АТ-В. 


Тһе matrix А represents the coordinates ау of the n points with respect to 
one set of m common-factor axes, say Pi, Fs, .. . , Fm; the matrix B repre- 
sents the coordinates b;, of the points with respect to a new set of axes, say 
Ky Ko,..., Kn; while the matrix T represents the transformation of the 
coordinates іп А to those in B. The foregoing transformation implies that 
the number of axes in the two reference systems is the same. Actually, how- 
ever, it is only required that the n points be contained in the same space of m 
dimensions, although the second reference system might employ a larger 
number of coordinate axes. 


If the number of factors is equal to the number of variables then the solu- 
tion for T is simply 


(13.2) T = А-В. 


Since the number of factors is usually much smaller than the number of 
variables, however, the matrix A does not have an inverse and T cannot be 
calculated directly. It may be noted that, for any matrix А, 


(13.3) (А'А)-ЧА'А) = I, 


and hence, if both members of (13.1) are premultiplied by (А'А)-!А', there 
results 


(13.4) Т = (A'A)J?2A'B. 


This formula gives the desired matrix of transformation. 
| The distinction between the problem treated here and the transforma- 
tions considered in Chapter X should be noted. In relating two solutions, 
the factor patterns (A and B) are assumed to be known, and the matrix of 
transformation (T) from one to the other must be determined. Inobtaining 
а derived solution, as in Chapter X, only the initial pattern (A) is given, 
and a matrix of transformation (T) is built up to produce the desired final 
solution (B). ы 
Тһе relationships among the factors themselves may also be obtained by 
means of the matrix T. Without loss of generality the column vectors of 


ї 


u 


RELATIONS BETWEEN DIFFERENT FACTOR SOLUTIONS 291 


the first and second sets of factors may be taken to be F = (Fi Р: F} and 
K = (K, К, Кз K,}, respectively. From the definitions of the two factor 
patterns, and their assumed equality, it follows that 


(13.5) AF — BK. 
Premultiplying both sides of this equation by (A'A)7!A', and again employ- 
ing (13.3), there results 
(13.6) F = (A'A)?A'BK. 
By making use of (13.4), this expression finally simplifies to 
(18.7) F-TK. 
This is the matrix formulation of the relationships between the factors P, 
Fo, F; and Ki, Ko, Ks, Ks. 
The important distinction between the relationships (13.1) and (18.7) can 


be exhibited clearly for the case of three factors іп the first solution and four 
factors in the second. The matrix T may be represented by 


а аса fi 


c2 do ез fe 
cs ds es fs 


(13.8) T- 


1 


in which the c's, d's, e's, and f’s are constants calculated by means of formula 
(13.4). From the matrix equation (13.1), four algebraie equations can be 
written for the expression of the coordinates of the second solution in terms 
of those of the first, as follows: 


аал + Свае + 6305s = ba, 
dian + Фар + заз = bis 
вал + егар + estjs = bjs, 
fian + fsajs + foais = dis, 


(13.9) 


Similarly, from the matrix equation (13.7), three algebraic equations for the 
expression of the first set of factors in terms of the second set can be written 
as follows: ' 
Е, = аК, + ФК» + Кз + fs, * 
(13.10) Е, = К, + dK: + еКз + ^К, 
Ез = К. + К + esKs + ЬҺК.. 


It may be noted that the transpose of the matrix of coefficients of (13.9) 
appears in (13.10). However, the variables in the former system are coordi- 
nates, while those in the latter are factors. 

& 


^ 
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А system of equations of the type (13.9) gives the actual transformation 
of coordinates between the two factorial solutions. Thus the factor weights 
for any variable in one solution are expressed linearly in terms of the weights 
of the other solution. Equations (13.9) may also be used as a check on the 
caleulation of the elements of T since the factor weights of both solutions 
are known. 

Ап alternative way of expressing the relationship between two factorial 
solutions is afforded by a system of equations of the type (13.10). Since all 
factors are assumed to be in standard form, the variances of the F’s can 
be simply calculated from (13.10) in terms of the variances of the К?з. For 
example, if the K’s are uncorrelated, 100c? is the per cent contribution of 
the factor K, to the variance of F,. Equations (13.10) might also be useful 
in estimating the factors F if the equations of estimation of the K's are 
known. 


13.3. Numerical Illustration 


A detailed numerical illustration of the procedures of the last section will 
now be given, employing the example of thirteen psychological tests. The 
relationships between the bi-factor solution of Table 6.12 and the oblique 
solution of Table 11.6 will be obtained. The matrix of coefficients of the bi- 
factor pattern will be denoted by B, while that of the oblique factor pattern 
will be denoted by A, in agreement with the general treatment of 13.2. For 
the convenience of the reader these matrices are repeated here: 


ҮІ Үг Үз Bo В, В, B; 
781 —.089  .142 ‚614 .425 
441  .004  .004 -339 .296 
721 —.090 —.142 .369 .475 
.508 .090 —.003 .460 .320 
—.058  .801  .087 .654 467 
097 .809 —.051 | 604 ‚526 
А = || —.068 .901 —.030 |, В = ||.578 :645 
1155 ,501 .078 | 718 235 
—.068 .919 —.081 || .532 .689 
—.385  .164  .809 .244 ‚708 
—.039  .077 .659 .423 .520 
073 —.177 .773 346 .641 
1351 —.061  .594 644 .429 
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The first problem is the determination of the matrix T which transforms 
A into B. This is given by formula (13.4). In applying this formula, the 
product A'A is first calculated. This may be done most simply by computing 
the sums of the squares of each column of А and the sums of the paired 
products of the columns two at a time, giving 


1.823 - .232 - .058 
A'A = | – 232 3.394 — -017 | $ 
— .058 — .017 2.103 | 


The next step involves the calculation of the inverse of the preceding 
matrix. The outline for the calculation of an inverse is given in Appendix 


TABLE 13.1 
FORWARD SOLUTION FOR CALCULATION. OF (A'A)-! 


Line | ү! Үз үз Identity Matrix 
a 1.823 - .932 - .058 1 0 0 
Be sits =i 127 .032 -.549 0 0 
&.. 3.394 = .017 0 1 0 
4.. - .029 — .007 .127 0 0 
ать TA 3.365 — .024 „1927 1 0 
бз өз шын =1 007 — .038 -.297 0 
7 | 2.103 0 0 1 
8 — .002 .032 0 0 
9 — .000 .001 007 0 
ТОР MAE, darin 2.101 .033 .007 1 
Ds esie Fosse een d жатан Jo. —.016 -.003 —.476 
Я | 


G.5, and the numerical details for the present case are exhibited in Table 
13.1. The inverse matrix is obtained by means of the back solution, em- 
ploying the values in lines 11, 6, and 2, as follows: 
| .554 .038 .016 

(МА)! = |.038 .297 .003 

016 .003 .476 


The check column is omitted in Table 13.1 because of the check afforded 
by the property of the product of a matrix by its inverse being the identity 
matrix, namely, 
1.000 .000 .001 

.000 .999 — .002 

.001 —.001 1.000 


(А'А)(А'А) = 
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After obtaining the inverse matrix it is postmultiplied by А, row by 
row,* which yields 


(WAA = 

.404.245 .394 .285 —.000 .050 —.004 .110 —.004 —.194 —.008 .046 .202 
002 018 .001.046 .236 .242 .265.182 270 037 .023 —.047 —.003 
079 .009 —.056 007 .043 —.021 —.013 .041 —.037 379 .313 369 288 


Finally, upon postmultiplying this matrix by B, the desired matrix of trans- 
formation is obtained: 

.808 .523  .047 —.024 
-757 .021  .637  .007 
.590 .012 —.015  .789 


T = (A'A)7A'B = 


Тһе actual rotation from the oblique coordinates to the bi-factor coordi- 
nates is given by 


ӛр = .808ал + 757 + .590а;з , 
ba =__.52Зал + .021а; + .012аз, 
ӛз Ез 047a; + .637aj; = .015a;; " 
bj = —.024aj + .007ау + .789a;; , 


(13.11) 


where the a's and b's are the coordinates in the matrices А and B, respec- 
tively. It is evident from these equations that the coefficients of the first 
bi-factor (general) are composed of fairly uniform portions of the corre- 
sponding coefficients of all three oblique factors. The coefficients of each 
of the remaining bi-factors are composed almost entirely of fractional parts 
of a single oblique factor coefficient. 


Similarly, by means of the matrix T, the relationships among the factors 
may be exhibited as follows: 


т = .808B, + .523B, + .047В, — .024В;, 
(13.12) үз = .757Bo + .021B, + .637В, + .007В; à 
үз = .590B, + .012B, — .015B; + .789B,. 


From these equations it appears that each of the oblique factors consists 
of a substantial portion of the general factor and the corresponding group 
factor of the bi-factor solution. For example, the factors By and В» con- 


* This is equivalent to the conventional row-by-column multiplication (A'A)-! by А’. 
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tribute 57 per cent and 41 per cent, respectively, to the unit variance of уз. 
It may be noted that the sum of the squares of the coefficients in the respec- 
tive equations of (13.12) is not exactly unity. This discrepancy is due to 
the fact that the assumption of the equality of the two factor patterns is 
met only roughly. That is to say, the communalities and the correlations 
reproduced by the two solutions are not quite the same; or, in geometric 
terms, the two sets of coordinates do not define the same points precisely. 


13.4. Suggestions for Order of Analysis 


From the various methods of analysis presented throughout the text it 
is evident that the choice of a particular form of solution is somewhat arbi- 
trary. In all cases, however, a certain routine of analysis must precede the 
final choice of solution. An effective outline which may be followed for all 
problems is given below. The first seven steps deal.with the preliminaries, 
While the eighth step for any problem is to select a particular form of the 
final solution. Following the outline there will be a brief recapitulation of 
the bases for the choice of a preferred solution. 


- Formulate the problem. 

- Decide upon a set of variables relevant to this problem. 

- Calculate all the product-moment correlation coefficients. For most rigorous 
results the distributions of the .variables should be investigated for normality 
апа the correlations tested for linearity. Reliability coefficients may also be 
helpful in determining the adequacy of the variables as effective instruments of 
measurement. 

4. By inspection, rearrange the variables into tentative subsets of highly interre- 
lated variables. 

5. If each variable in a subset has high positive correlations with all other variables 
in this group, and negative correlations with the remaining variables, reverse 
the scale of measurement for each of these variables and thus obtain positive 
correlations throughout. 

6'. If all the correlations are essentially 


омы 


6". If, after the reflection of variables, 


positive, the method of B-coeffi- 
cients can be employed as an objec- 
tive procedure for determining dis- 
tinct subsets of variables. The num- 
ber of groups roughly determines the 
rank of the correlation matrix. 


there are still negative correlations, 
then the method of B-coefficients 
does not apply, and an approxima- 
tion to the rank of the correlation 
matrix is not obtainable at this 
stage. Similarly, when a matrix con- 
tains positive correlations through- 
out but does not indicate distinct 
clusters of variables, then the rank 
cannot be approximated by the 
method of B-coefficients. 
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7’. Calculate complete estimates of the 


communalities of the variables, as 
described in 7.2, by the direct meth- 
ods when the rank is one or two, or 
by the bi-factor method for higher 
rank. Then obtain a centroid solu- 
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7". When complete estimates of the 


communalities are not available, 
some arbitrary choice of the com- 
munalities must be made. One of the 
simplest methods is to select the 
average of all the correlations for 


tion based upon these communali- each variable as its communality. 
ties. This gives rise to the averoid solu- 
tion described in 8.4. 


8a. Calculate a bi-factor solution as the final form by the method described in Ap- 
pendix C. 


8b. Calculate a principal-factor solution as the final form. For a small number of 
variables this may be obtained directly, when the communalities are known, by 
the method of Appendix D. Otherwise, the principal-factor solution can be ob- 
tained by transformation from an initial solution as described in Chapter IX. 


8c. Caleulate a multiple-factor solution as the final form by the methods described 
in Chapter X. 


8d. Calculate an oblique solution as the final form by the methods described in 
Chapter XI. 


When the analyst has reached Step 8 in the fore 
need to consider the bases for choice of one of the p 
tion. As indicated throughout the book—especially 
XI—such a choice depends upon certain statistical c 

the nature of the problem and the data. If the inv 
theory in a given field, such as Spearman's theory of intelligence, then he 
might select the particular form of factor solution which would verify the 
theory. In another type of problem the variables may be of such a nature 
that the bipolar form of expression of the factors may appear most simple 
and clear to the analyst. An example of this is given by the solution of eight 
physical variables which is presented in Table 7.4. An eminent biologist 
selected the principal-factor form not only because it was the most statisti- 
cally elegant solution but also because the bipolar nature of the factor 
“body type" was explicitly revealed. For other problems the chief aim may 
be to secure the greatest simplicity in the factorial solution. Usually this 
may be accomplished by the choice of an oblique factor pattern, as illus- 
trated in Chapter XI. Certain data, however, such as the political variables 
analyzed in 11.5, may yield an oblique solution which is no simpler than an 
orthogonal one. For such data, then, an orthogonal solution would be pre- 
ferred. In other cases, also, if the investigator prefers a solution with un- 
correlated factors, he would select the bi-factor form including a general 
factor, or the multiple-factor form involving group factors only. 


going outline, he will 
referred forms of solu- 
in Chapters I, V, and 
riteria as well as upon 
estigator starts with a 
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It should be evident that in the choice of the form of solution the analyst 
must be guided both by the statistical criteria and by other considerations, 
such as theories in given fields and the nature of the data being analyzed. 
Since the interdependence of criteria and data will vary with the type of 
problem, the choice of the analyst will vary in different situations. Such 
variations, however, are due largely to the convenience of a particular form 
of solution for a given body of data and not to any essential uniqueness of 
that solution. The fact that all solutions for a given problem may be 
brought into relationship with one another indicates that in factor analysis, 
as in other branches of applied science, severa! equally satisfactory laws 
may be employed as expressions of the essential nature of the phenomena 
being studied. 


APPENDIXES 


APPENDIX А 
FUNDAMENTALS OF MATRIX THEORY 


It is the aim of this appendix to present in brief outline form the funda- 
mental definitions and theorems on matrices.* First, however, & short re- 
view of determinants will be given. 


1. Definition of a Determinant of Order 2 

А sum of product terms, with alternating algebraic signs, frequently oc- 
curs in mathematical work and hence has been given а special notation. 
For example, the expression 


ad — bc 
is denoted by the symbol 

a b 

e d|' 


which is called a determinant of the second order, since it contains two rows 
and two columns. 


2. Definition of a Determinant of Order 3 


The symbol 
а\ ы Сі 
аз b C 
аз b3 Cs 


is called a determinant of the third order and stands for 
аса — аас + абас — азса + aabsce — asbiei . 


A method for obtaining the expanded form of а determinant will be de- 


Scribed in 5 below. . 
The nine numbers аз, . . . сз are called the elements of the determinant. 


In the symbol these elements lie in three (horizontal) rows and also in three 


* For a detailed treatment see Bocher, Introduction {о Higher Algebra, pp. 20-80; 
Dickson, Modern Algebraic Theories, pp. 39-63. 
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(vertical) columns. For example, аз, Әз, cs are the elements of the third row, 
while the three b's are the elements of the second column. The diagonal 
from the upper left-hand corner to the lower right-hand corner is called the 
principal diagonal. Yn the third-order determinant the principal diagonal 
contains the elements ai, be, сз. 


3. Definition of a Determinant of Order n 


A determinant of general order n may be denoted by 


ай аз ... аһ 

ат а» ... аж 
A= " 

ал аз ... Onn 


where the n? elements are denoted by a’s with two subscripts, the first repre- 
senting the number of the row and the second the number of the column in 
which the element appears. By definition the determinant A shall stand for 
the sum of the n! terms each of which is (apart from sign) the product of 
n elements, one and only one from each column, and one and only one from 
each row. The algebraic signs are determined most easily by the method 
of expanding the determinant which is explained in 5 below. 


4. Minors and Cofactors 


The determinant of order n — 1 obtained by striking out the row and 
column crossing at a given element of a determinant of order n is called the 
minor of that element. Thus, corresponding to the element аҙ, which stands 
in the jth row and the kth column of the determinant A, there exists the 
minor Му» which is obtained upon crossing out the given row and column. 
Frequently there is occasion to consider not this minor M ;, but the cofactor 
A;r of a;, defined by 

Аж жə (-Ц%%М, . 
The algebraic signs attached to the minors to obtain the corresponding 


cofactors are alternately + and —, as indicated by the following diagram 
which is associated with the elements of a determinant: 


+-+- 
-+-+ 
+-+- 


——— 
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5. Expansion of a Determinant 
Any determinant А may be expanded according to the elements of any 
row or any column, that is, 


A = M anA (j 53,2, 2, ny, 
k=1 

А = Sans (Е =1,2,...,7). 
ј=1 


Thus, for the third-order determinant 


Qu (із (із 
А = |an аз аз), 
аз (32 @зз 


the expansion according to the elements of the second column becomes 


А = andis + aAa + 032A 3 
= —авМ» + aM? — (Ма 


-ағ(аФдазз — алазз) + азз(апазз - азан) — @з°(йпазз — атам), 


Ш 


which, upon rearranging of terms, шау be written as follows: 


А = апаззазз + 01202301 + 013032021 — (013022031 — 93032011 — Q33021012 . 


By successive application of the method of expansion of а determinant 
according to the elements of some row or column, а determinant of any order 
eventually can be reduced to the explicit expansion of determinants of the 


Second order. 


6. Definition of a Matrix 

A system of mn numbers а;к arranged in а rectangular array of m rows 
and n columns is called an m X n matriz. If m = n, the array 18 called 
a square matrix of order n. A matrix will generally be represented as follows: 


ал (ю ... (із 

Qs 49... (т 
А- , 

аһ Am ... Amn 


although parentheses are sometimes used instead of the double bars, or the 
matrix might be designated by its general element in the form (|ауь|| or 
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(a;,). The notation here used, matrices being denoted by bold-face type, is 
systematically followed in the text. 

Simple examples of one-rowed matrices are the notations (т, y) and 
(z, y, 2) for points in а plane and in space, the elements of these matrices 
being the coordinates of the points. Similarly, the set of coordinates repre- 
senting three points in а plane, when arranged in a definite order, form a 
matrix, e.g., 


7. Distinction between Matrices and Determinants 


It should be noted that even when a matrix is square it is not a determi- 
nant. А determinant, whose elements are real numbers, represents a real 
number, while а matrix does not have a value in the ordinary sense. The 
difference between a square matrix and a determinant is clearly seen upon 
interchanging the rows and columns; the determinant has the same value, 
but the matrix is generally different from the original one. 


8. Transpose of a Matrix 


А matrix which is derived from another by interchanging the rows and 
columns is called the transpose of the original matrix. Thus, if 


ай а; аз 
А = |an аз os 
Qa G32 Q33 


then the transpose of А is the matrix 


Gi ап аз 
A’ = |а: аз аз 


(із Q23 азз 


The prime notation for the transpose is followed throughout the text. 


9. Determinants of a Matrix 


Although square matrices and determinants are wholly different things, 
it is possible to form from the elements of a square matrix ‘a determinant 
which is called the determinant of the matrix. The notation which is em- 
ployed is bold-face type for the matrix and the corresponding italic type 


——————— С етеш 
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for the determinant of the matrix. Thus the determinant of а square ma- 
trix А is denoted by А. Other determinants, of lower order, can be formed 
from any rectangular matrix by striking out certain rows and columns. For 
many problems it is important to know the order of the highest nonvanish- 
ing determinant of a matrix. 


10. Rank of a Matrix 


A matrix A is said to be of rank r if it contains at least one r-rowed deter- 
minant which is not zero, whereas all determinants of A of order higher than 
т are zero. 

By the rank of a determinant is meant the rank of its matrix. 


11. Singular Matrix 


A square matrix is said to be singular if its determinant is zero. Other- 
wise, it is called nonsingular. 


12. Matrix Equations 


Any two matrices A and B are said to be equal if and only if every ele- 
ment of A is equal to the corresponding element of В. Thus, if A = (ау) 
and В = (5) then the equation 


A=B 


implies that aj, = bj. for every j and k. Thus it is evident that a single ma- 
trix equation stands for as many algebraic equations as there are elements 
in either of the matrices which are equated. 


13. Symmetric Matrix 


A matrix A is symmetric if and only if it is equal to its transpose A’. In 
other words, the matrix A = (ajx) is symmetric in case it remains unaltered 
by the interchange of its rows and columns, i.e., 


ак = ак; (k212...,9. 


Тһе following is an example of a symmetric matrix: 


78 —16 .23 .04 
—16 .59 —.34 —.21 
93 —.34 86 .40 
04 —.21  .40 .65 
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14. Sum or Difference of Matrices 

Тһе sum (or difference) of two matrices each of m rows and n columns is 
defined to be an m X n matrix each of whose elements is the sum (or differ- 
ence) of the corresponding elements of the given matrices. АП the laws of 
ordinary algebra hold for the addition or subtraction of matrices. 


15. Multiplication of Matrices 
The element in the jth row and the kth column of the product of a ma- 
trix A with n columns by a matrix B with n rows is the sum of the products 


of the successive elements of the jth row of A by the corresponding elements 
of the kth column of B. 


For example, if 


bu bis 
A= ап а (із В-|ьы bas 
ат аж аш)! i 
21 зз Олз he Бап 
then the product С of these matrices is » 
С-А.В- ubi + аза + азба anbis + алго + arabo 
азди + азба + азба аз +. абз + azaba 


It should be noted that in this row- 
the number of columns in the first ma 
rows in the second. The product matrix then contains the number of rows 
of the first matrix and the number of columns of the second. Thus, in the 
example, the product of the 2 X 3 matrix by the 3 X 2 matrix is a 2 X 2 


matrix. This may be conveniently noted by writing the order of each ma- 
trix as superscripts, namely, 


by-column multiplication of matrices 
trix must be equal to the number of 


A2x3 è взх? = С?х? " 
In general, 


А". Baxs Стхе ; 


that is, the product of an m X n matrix by an n X s matrix 
matrix. 


Multiplication of matrices is not commutative in general, that is, 


AB x ВА. 


isanm xX s 


Hence it is important to specify in what order matrices а 
the product AB the matrix B is said to be premultiplied b; 
A is postmultiplied by B. 


re multiplied. In 
y the matrix A, or 
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16. Scalars 


In order to distinguish the ordinary quantities of algebra (i.e., real and 
complex numbers) from matrices, the former are called scalars and will here 
be designated in italics. The product of a matrix A by a scalar k (kA or Ak) 
is defined to be the matrix each of whose elements is k times the correspond- 
ing element of A. АП the laws of ordinary algebra hold for the multiplica- 
tion of matrices by scalars. 


17. Diagonal and Scalar Matrices 


А matrix in which the diagonal elements do not all vanish and all remain- 
ing elements are zero is called a diagonal matriz. A special instance of such 
a matrix is one in which all the elements of the diagonal are identical; it is 
then called a scalar matriz. If а scalar matrix 


k © ... 0 

` О dE «xa 0 
k= 

00... k 


is premultiplied or postmultiplied by any matrix A of the same order as k, 
the following relationships become evident: 


kA = АК = КА. 
In particular, the matrix 
Y 0. 0 
01-1. 0 
ic 
i OW sce T 


is called the identity matriz, and it has the property that, if A is any matrix 
whatever, 
ІА = АІ = А. 


It is evident that, in matrix algebra, all scalar matrices may be replaced 

Y the corresponding scalars and, conversely, that all scalars may be con- 

Sidered as standing for the corresponding scalar matrices. The identity ma- 

trix I corresponds to unity in ordinary algebra, and hence in products of 
Matrices the factor I may be suppressed. 
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18. Inverse Matrix 


If a matrix 
Qu аі а 
алі ал? don 
A= 
Qni GQn2 ... (тт 


is nonsingular, i.e., А > 0, then there exists another matrix 


An An Am 
a T Яң йз se dm 
Acus ra a èks VE EN Аз Ag ... йш 
m АД uox o em ә» 
Аһ Аһ Ам қайды 

А A `? A 


in which A+; denote the cofactors of the elements of A. The matrix А-! is 


called the inverse of A and is itself a nonsingular matrix which has the prop- 
erty 


AA? = А-А — I. 


It should be noted that the rows and columns of cofactors in the inverse of 
a matrix are interchanged. In other words, when 1/A is factored out, the 
element A,; in the jth row and kth column of the inverse of А is the cofactor 
of the element as; in the kth row and jth column of A. 


19. Theorems on Transpose and Inverse of Products of Matrices 


The transpose of a product of matrices is equal to the product of their 
transposes taken in reverse order. Thus, 


(АВС) = C'B'A'. 


The inverse of a product of matrices is the product of their inverses taken 
in reverse order. For example, 


(ABC) = С-1В-!А-!, 


APPENDIX В 
MISCELLANEOUS TOPICS AND PROOFS 
В.1. Brief Description of a Set of Twenty-four Psychological Tests 


To illustrate the factorial methods a numerical example is introduced 
in 2.8. The variables from which the fundamental data (the correlations of 
Table 2.2) are obtained consists of twenty-four psychological tests which 
were given to № = 145 children of the Grant-White School of Forest Park, 
Illinois.* Copies of the tests are reproduced in the monograph by Holzinger 
and Swineford, so that only brief descriptions of them are given here. 


1. Visual Perception Test. A nonlanguage multiple-choice test composed of items 
selected from Spearman’s Visual Perception Test, Part III. Testing time: 19 min- + 
utes. 

2. Cubes. А simplification of Brigham's test of spatial relations. Testing time: 
8 minutes. . 

3. Paper Form Board. А revised multiple-choice test of spatial imagery, with 
dissected squares, triangles, hexagons, and trapezoids. Testing time: 8 minutes. 

4. Flags. Adapted from a test by Thurstone. Requires visual imagery in two or 
three dimensions. Testing time: 53 minutes. 

5. General Information. A multiple-choice test of a wide variety of simple scien- 
tific and social facts. Testing time: 18 minutes. 

6. Paragraph Comprehension. Part III of Traxler Silent Reading Test, Form 1, 
for Grades VII-X. Comprehension measured by completion and multiple-choice 
Questions. Testing time: 20 minutes. 

7. Sentence Completion. A multiple-choice test in which “correct” answers reflect 
good judgment on the part of the subject. Testing time: 6 minutes. 

8. Word Classification. Arranged by M. A. Wenger. Sets of five words one of 
Which is to be indicated as not belonging with the other four. Testing time: 10 
minutes, 

9. Word Meaning. Part 11 of Traxler Silent Reading Test. A multiple-choice 
Vocabulary test. Testing time: 14 minutes. 

10. Add. Speed of adding pairs of one-digit numbers. Testing time: 2 minutes. 

11. Code. A simple code of three characters is presented and exercise therein 
given to measure perccptual speed. Testing time: 2 minutes. 

12. Counting Groups of Dots. Four to seven dots, arranged in random patterns, 
to be counted by the subject. A test of perceptual speed. Testing time: 4 minutes. 

13. Straight and Curved Capitals. A series of capital letters. The subject is re- 
Чшге to distinguish between those composed of straight lines only and those con- 
taining curved lines. A test of perceptual speed. Testing time: 3 minutes. 


“кр Holzinger and F. Swineford, А Study in Factor Analysis, 1939. 
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14. Word Recognition. Twenty-five four-letter words are studied for three min- 
utes. These words are then to be checked from memory on a hundred-word list. 
Testing time: 5 minutes. (Score includes two forms.) 

15. Number Recognition. Similar to Test 14. Fifteen three-digit numbers. 

16. Figure Recognition. Similar to Test 14. Fifteen geometric designs. 

17. Object-Number. Twenty pairs of names of familiar objects and two-digit num- 
bers are studied for three minutes. The words only are then presented to the sub- 
ject, who is required to supply the proper numbers. Testing time: 5 minutes. (Score 
includes two forms.) 1 

18. Number-Figure. Similar to Test 17. Ten pairs of numbers and geometric 
figures. 

19. Figure-Word. Similar to Test 17. Ten pairs of geometric figures and words 
Studied for one minute. 

20. Deduction. Logical deduction test using the symbols ) and ( and the letters 
А, B, C, and D. Testing time: 24 minutes. 

21. Numerical Puzzles. A numerical deduction test, the object being to supply 
four numbers which will produce four given answers employing the operations of 
addition, multiplication, or division. Testing time: 14 minutes. 

22. Problem Reasoning. A reasoning test in completion form. Each problem lists 
the steps in obtaining a required amount of water using two or three 
capacity. Testing time: 14 minutes. 

23. Series Completion. From a series of five numbers the subject is supposed to 
deduce the rule of procedure from one number to the next, and thus supply the sixth 
number in the series. Testing time: 14 minutes. 

24. Woody-McCall Mixed Fundamentals: Form I. А series of 35 arithmetic prob- 
lems, graduated for difficulty, is included. Testing time: 20 minutes. 


vessels of given 


It should be noted that Tests 3 and 4 are Tests 25 and 26 of А Study in 
Factor Analysis, while Tests 3 and 4 of that monograph have not been used 
in the present study, This change has been made as a consequence of the 
findings in the other study. It was found that the original Tests 3 and 4 
were too difficult for the group of children in the sample and hence yielded 


smaller reliability coefficients than the revised forms 25 and 26 (which are 
called here Tests 3 and 4). 


В.2. Factor Palterns as Classical Regression Equations 


It will be shown here that any equation of a factor pattern may be con- 
sidered as a classical regression equation, where the dependent variable is 
the observed variable and the independent variables are the hypothetical 
factors. The formal expression for such an equation is 


(B.1) 2) = Bal + Bifa +... + BinFm + BLU; 
GS hehe. uD; 
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where the subscripts on £ are so arranged that the position of any regression 
coefficient is uniquely determined. Thus, for the coefficients Bis, the first 
subscript j indicates that z; is the dependent variable, and the second sub- 
script s (s = 1,2,..., m) shows the factor F, to which the coefficient is 
attached. The regression coefficient attached to the unique factor U; is dis- 
tinguished merely by one subscript. Of course, (B.1) cannot be obtained 
by the ordinary methods of solution for a regression equation because the 
factors are not observed variables. It will be shown, however, that a pat- 
tern produced by a factor analysis consists of equations of the form (B.1). 

The formal expressions for the coefficients in (B.1) may be written, al- 
though the factors are not observed variables. The correlations among the 
variables involved in (B.1) may be given a symbolic representation even 
though they may not be obtainable by experiment. Let the matrix of inter- 
correlations of factors, bordered by the correlations of any variable z; with 


the factors, be denoted by A. Then 4 
1 Тр, Тык, ТР, ++. Тағ, Там; 
LT 1 Тыл, Тө” tins Te, 0 
Tr,2; ТЕ,Е, 1 Тю, ә. Жр 0 ‚ 
А = || Tr; Tre TFF, 1 exces HERE. 0 |, 

| $ 

Tey Thaly ҮТЕР, TESEQ «4» 1 0 

L Тор) 0 0 0 0 


since the unique factors are assumed to be uncorrelated with the common 
factors. 

Let Ai be the minor of the element in the first row and column, A; the 
cofactor of the element Tu; and Aj, the cofactor of the element Tur, 
(s = 1, 2,..., m). Then the regression coefficients in the linear equation 
for z; are given by* 


А; 

В, ЗЕРЕ 1 

(В.2) В, x 
and 

| А; 

| (B.3) mE "n 


* Karl J. Holzinger, Statistical Methods for Students in Education, p. 313. 
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The last expression can be simplified by inserting the value A; = —r. ju, An. 
It then becomes 
В; = т.о; 
ог, оп applying formula (2.18), 
(В.4) В; = а;. 


The algebraic solution, by determinants, for the coefficients а; of the 
factor pattern, 


(2.45) Zi = аһ + аР +... + ajmFm + ajU;, 


by means of equations (2.17) is* 


(2.195) as = 4 (= 1,2) 060,09 @ 1,2, Vm). 


Now А is precisely the determinant An, and А,, is readily seen to be —Aj. 
Hence 


(B.5) Bj = aj. 


Formulas (B.5) and (B.4) show that the equations (2.16) of a factor pat- 
tern are exactly of the form of the classical regression equation (B.1). 

It will next be shown that the equations of a factor pattern, considered 
аз regression equations, include the errors of estimate. In other words, the 
multiple correlation between the dependent variable z; and the independent 
variables Fi, Рҙ,..., Fm, О; is unity. The standard error of estimate of 
the variable 2; from the regression equation (B.1) is given byt 


(B.6) ENC 


Expanding the determinant A according to the elements of the last column, 
equation (B.6) may be written 


A -тіл;Ап + (ân + Тұң An + Tar, Ajat nnn H Тағ, А) 
5? = = . , 
Ап An 


which simplifies to 
(B.7) 8-1- (rir, Bà T тан Өл “Б... Тағы а) — ай 
upon substituting the values from (В.2) and Tau; = йу. 


* бее р. 16. 1 Holzinger, ор. cit. 


ee 
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In (B.5) the relationship 8, = aj, was deduced, and for uncorrelated 
factors r:r, = aj as shown in (2.20). Putting these values into (B.7), it 
reduces to 


(В8) 8 = 1 (а +а, +... + а,) – а = 1 № – а = 0, 


where Àj is the communality of variable j, and the last equality follows from 
the fact that the communality and uniqueness sum to the total unit variance 
of a variable. The last equation shows that, in predicting a variable z; from 
the independent variables Г}, Рз, . . . , Fm, U;, there is no error of estimate. 
А factor analysis usually produces the values of a;, so that the communality 
hj is determined. The coefficient of the unique factor U; is then obtained 


merely from the relation, 
a; = Vl—H, 


to bring the total variance of the variable up to unity. Thus any equation 
of a factor pattern may be thought of as a classical regression equation with 
the error of estimate given by the coefficient of the unique factor. 


В.3. Indeterminateness of Factor Solutions 

It is a well-known proposition that the factor problem is indeterminate 
in the sense that, given a matrix of correlations of a set of variables, the 
coefficients of a factor pattern are not uniquely determined. That is, sys- 
tems of orthogonal, or uncorrelated, factors F; may be chosen, consistent 
with the observed correlations, in an infinity of ways. Perhaps the clearest 
demonstration of this theorem was given by Hotelling,* and that is the 
form that will be employed here. 

For simplicity of proof it will be assumed that the factor pattern con- 
tains as many factors as variables. Without specializing the type of factors 
(e.g., into common and unique), the pattern may be written 


(B.9) 2; = any + ila +... + ass Fs (= 1,2) 00452) 5 


where the factors are normally distributed having zero correlations and in 
Standard form, that is, zero means and unit variances.] The condition that 
the means shall be zero is expressed by 


_ =F, _ 
SN 


* Harold Hotelling, “Analysis of а Complex of Statistical Variables," Journal of Edu- 
cational Psychology, 1933, Sec. 1. 

T It will be assumed here that these properties, which are postulated for an infinite 
Population, also hold for & sample of N observations. 


(B.10) М» 0, 
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where the summation is taken over N values. The properties of zero corre- 
lations and unit variances are expressed in the single statement 


(B.11) I Las, 


where б, is the Kronecker delta, which is equal to unity if s = t and equal 
to zero if s # t. 

Now the correlation between variables z; and 21, equal to unity if j = k, 
is given by 


Sz. 
(В.12) т = TER, 
where the summation is over the N values of the variables z; and 2,. Sub- 
stituting the value of 2; from (B.9) and a similar expression for 2, into 
(B.12), and making use of the conditions (B.11), equation (B.12) becomes 


(B.13) rik = > ама, (,8-1,2,...,т). 
8-1 


Тһе indices j and k take on л? values together,* but since гук = ть; the 
number of equations (B.13) is only 


nt - (9) =- ma-a. 


This number of equations is insufficient for uniquely determining the n? 
quantities a;, when the correlations are given. The difference between the 
number of unknowns and the number of conditions is 


n? — n(n + 1) = n(n — 1). 


This suggests that it should be possible to express all the coefficients in 
terms of $n(n — 1) of them, or generally, in terms of 3n(n — 1) other 
parameters. Thus systems of uncorrelated factors F may be chosen, con- 
sistently with the observed correlations, in n(n — 1) ways. This result 
may also be interpreted geometrically. In a space of n dimensions there 
are yn(n — 1) degrees of freedom of a rigid rotation. Thus after a set of 
n factors is selected there are 3n(n — 1) degrees of freedom of a rigid rota- 
tion of this set into another set of factors. 

One assumption made on the nature of the factors F has not been brought 


* It is here assumed that the diagonal values of the correlation matrix are unity. 


| 
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explicitly into the proof. That assumption is that the factors are normally 
distributed. It is because of this condition that no additional equations for 
determining the a;,'s could be obtained from the moments of higher order, 
or of other parameters of the population. For the г/в, being linear combi- 
nations of normally distributed factors, have a multivariate normal dis- 
tribution; and every parameter of such a distribution is a function of the 
means, standard deviations, and correlations.* Thus all the available in- 
formation is embodied in the equations (B.13), and these being fewer in 
number than the coefficients ау, the indeterminacy exists. 


B.4. Basic Axioms for Euclidean Geometry 


The most fundamental axioms are those of incidence or connection. They 
may be listed as follows for three-dimensional space: 


l. Any two distinct points uniquely determine a straight line. 

. If P, and Р» are distinct points, there is at least one point not on the straight 
line PiP». 

3. Any three noncollinear points determine a plane. 

4. If two distinct points Pi, Р» both belong to a plane т, every point of the straight 
line P,P: belongs to т. 

. If Pi, Ps, Ps are noncollinear points, there is at least one point not on the plane 
P;P2P3. 

6. If two planes have a point P; in common, they have a second point P: іп com- 

mon, and hence the line P:P2 in common. 


t2 


сл 


Axioms 2 and 5 postulate the existence of a two- and three-dimensional 
space, respectively. If Axiom 6 is assumed, it limits.space to three dimen- 
sions. On the other hand, if at least one point not in this region is postulat- 
ed, then а four-dimensional region is obtained. 

There is some difficulty in determining all the elements of space by means 
of the preceding axioms; for, while Axiom 1 postulates that two distinct 
points determine a line, there is no axiom guaranteeing that two distinct 
lines in a plane determine a point. This, of course, is not true in the Eu- 
clidean plane since parallel lines have no point in common. To simplify the 
geometric discussion, therefore, the following projective axiom will be as- 
sumed: 

7. Any two distinct straight lines in a plane uniquely determine a point. 
This axiom extends a Euclidean to a projective plane. The advantage of 


working in a projective space is that most theorems are greatly simplified 
by not having to treat the special cases of parallel elements. 


* Yule and Kendall, An Introduction to the Theory of Statistics, 1937, Chap. XIV, 
See. 28. 
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В.Б. Proof of Theorem 8.1 


For purposes of reference, the theorem is here repeated: If m is the rank 
of the matriz 


Тү Ti 28 ... Тін 

оі Ta Хз ... Ton 
[5:1 = | т Та Tag ... ©з), 

Тө Tng Tng ... бау 


the points Py, Po,..., Р, are all dependent upon т of them, which are them- 
selves independent. 

The proof of this theorem may be split into two parts. First consider the 
case where n € N. By hypothesis the matrix || is of rank т, во that 
without loss of generality it may be assumed that the determinant 


Тп Tiz Tim 
pa Ta X29 Тот 
Umi TIm? ... Тат 


is different from zero. If m — n the set of equations 


n 


Sate = 0 6 =1,2,...,n) 
k=1 
have the unique solution һ ==... = 1, = 0, since D # 0. Then, ac- 


cording to the definition (3.2), the points Pi, Pa, ... . , P, are linearly inde- 
pendent. If m < n the points Pi, Pz, . . . , P, may be shown to be inde- 


pendent by the preceding argument. This establishes the last part of the 
theorem. 


Now it remains to be shown that all л points are de 


pendent upon these. 
Let Di, Dai, . . . , Dmi be the cofactors of Tii, Loi, 


+» mi in the matrix 


Ti 339 sos Uu Tu 
Ті 2ш... Топ То) 

Ж М 
Tmi Тм? Imm Tmi 


Ты Жә ... Фт Xy 
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where p = m 4- 1,..., n and i is arbitrary. The determinant of this ma- 
trix, when expanded according to the elements of the last column, becomes 


(B.14) хри + рн +... + EmiDmi + ты). 


This expression vanishes, for, if 7 € m, two columns then have equal ele- 
ments; and, if 7 > m, it vanishes, since the rank of [Iz;;| is m and every 
(m + 1)-order minor vanishes. The solution for ть; from the expression 
(B.14) set equal to zero is 


(B.15) LP (р-т-1,...,т), 
4-1 

where the constants 

TEE 

а == D 
do not depend on the elements zi; ze; . . . , tni, tpi. It follows from defini- 
tion (3.1) that the points Pp, whose coordinates are given in (B.15), are 
linearly dependent on the points Pi, Ps, . . . , Pm, which are themselves in- 
dependent. 

If n > М, consider the points P;: (zi, »,..., tjv, 0,..., 0) in the 


Space of т dimensions. Then the foregoing argument can be applied to 
obtain the relation (B.15), and thus the theorem is established for all 


values of n. 


B.6. Proof of Theorem 8.4 

The product of the matrix ||z;:|| of standardized values by its transpose 
is equal to the correlation matrix ||r;,|| multiplied by М, as shown in equa- 
tion (3.44). Since the theorem is concerned only with the ranks of the 
matrices, the non-zero factor N is irrelevant to it. The product of the two 
matrices might be represented by ||r*,|| = ||Nr;;|| to be absolutely rigorous, 
but for simplicity ||r*,|| will be written |ға|, With this clearly in mind, 
the theorem may be restated as follows: If m is the rank of the matrix 
|8, |, and if |24 | is its transpose, then the rank и of |Iril| = (2: zal 
is equal to m. 

The proof consists of two parts: first, to show that и € m and, second, 
that m < и. That the rank of the product of any two matrices cannot ex- 
ceed the rank of either factor is a well-known theorem. The remainder of 


1 See, e.g., L. E. Dickson, Modern Algebraic Theories, p. 51. 
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the proof is concerned with showing that m < и for the product of a matrix 


by its transpose. 
Let 


121 = 


2т1 


be a minor of |2; | whose rows form a maximal linearly independent set of 
rows of |z;|. By this is meant that the rows of 12,11 are linearly inde- 
pendent and every minor of which |2, | is а proper minor has its rows 
linearly dependent. Then the rank of |[z;|| is the number, m, of vectors 


(or rows) in this set. 


212 ... Zim 
252 ... Zam 
22 ... Zmm 


The product of ||,,|| by its transpose, ||z,.||, may be represented by 


11 = 


where ||r,,|| is ап m-order minor of 


Тп Та 

т т 
(В.16) e um 

Tmi Тт? 


then, premultiplying by |в z; . . 


0- а, 22... Zall 4 ШІ) ы 


(B.17) 


* This condition is equivalent to (3.2), 


ЕМЕСЕ 


llr;ll. Now, if* 


21 
Тіт 22 
Tom 
=0 ; 
Ттт 
| 2т 


- Z,||, this expression becomes 


21 
22 


Zm 


= 2.. 


ЕЕЕ lle. 


and if it implies thatz, = z; =... 


then the columns of the correlation matrix are linearly independent. 


2\ 
23 


= а EE -% 
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Let the product of the last two matrices be represented by the column vec- 
tor {y,}, as follows: 


n 21 
y» 22 

= |lecsll - А 
Ут 2т 


then the transpose of {y,} is 


ІМ = Wyn Y2- - ум] = Пе 22... 21 + 12411. 
Equation (B.17) may now be put in the form 
Ui 
уа 
т 
0 = lil + {0} = ln ge vl =|] | = УМ. 
ч E 
Ут 


In order for a sum of positive numbers to be zero, each element must be 
zero, and therefore |у) || is а zero vector. Then 


ЕЛ! = la 2%... ғһ|| ы (Б =0, 


and, since the rows of |jzpq|| are linearly independent, it follows from the 
definition (3.2) that zı = ze =... = 2 = 0, that is, the vector |a 22... nll 
is a zero vector. It has thus been shown that the condition (B.16) implies 
that а = 2; =... = Zm = 0, and hence according to (3.2) the columns of 
|» are linearly independent. Then the matrix |ғы) is nonsingular, and 
its rank и is at least equal to its order, m. Now that » = mand, as has been 
Pointed out, и < т, it therefore follows that и = m. The rank of the prod- 
uct of a matrix by its transpose has thus been shown to be equal to the com- 


шоп rank of either factor. 


В.7. Standard Error of a Function in Terms of the Standard Errors of Tis 


Variables 
Let ті, 2,..., tn) be a function of the n variables DI ERE 
let f; = 97/0, с; = Сер Tjk Тар Then the total differential is given by 


df = fidzi + fadzs +... + Јат. 


.,v,8nd 
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Squaring this expression and taking the mean value over the population 
produces 


оў = fici + fioi +... + Siok + (fifaro 
+ Л)збібгіз "Pasos + Лост + fofso2care3 
+... t+ Љоса +. ~~ + Јајда лса), 


= >, Лод +2 > Fifa jour jx, 
гі 


j<k=1 
non 
(B.18) оў= > > Sifno jour к, 
ізі k=1 
since rj; = land rjj = тк. 
For the particular case in which the variables 21, £2, . . . , 2, are statisti- 


cally independent (i.e., uncorrelated), formula (B.18) reduces to 


n 


(B.19) cj = >> fiet. 


ізі 


А very important special case of (B.18) occurs when f is a function of а 
single variable z. Then 


(B.20) до = ( а 


> == 


This states that the mean square error of a function of x is equal to the mean 
Square error of 2 multiplied by the square of the derivative of the function 


with respect, to т. It is this theorem which is employed in obtaining (6.25) 
or (6.26). 


B.8. Mean Value of Certain Ratios 


The mean value of a nonlinear function of two or more variables can, in 
general, only be expressed in terms of the means, standard deviations, and 
correlations of the original variables to a first approximation, Thus, let it be 
required to find the mean value of the ratio ab/c* in terms of these statistics 
for a, b, and c. The determination of the other four expressions in (6.28) 
follows a similar pattern to the one which will be treated in detail here. To 
save space, therefore, their derivations will not be given. 
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— n—1 = us А 
Denoting the mean over v = ( 2 ) values by scoring, it is required to 


find an expression for X(r,ri/ris; J < k, Jk se, Jk = 1, 2,...,m) fv, 
or employing the definitions (6.19), 


(B.21) АТТА м. 

с y mt ct 
where the summation extends over the > values of the ratio. Let a, 8, and y 
stand for the respective deviates of a, b, and c from their means, that is, 


а-а-а, B-b-—b, y-2c—t. 


Then (B.21) may be expressed as follows: 


а) _ 1 N^ ab a В n 
(а-а Аа. 

Now expand the last bracket by the binomial theorem, dropping third and 
higher degree terms. To this approximation, the formula becomes 


ab\ _ ab 1 ( a , B , а ( Y ye 
| == - 1+-+= + = — 3- Pd 
( ) RP» +++ ag 31 +55). 
This may be expanded, again keeping only terms to the second degree, as 
follows: 


D. xc 3 B ec 
Zy — Š Xoy — ELSE 
ac b с 


Employing the definitions of the standard deviation and the correlation 
coefficient for these variables, the mean of the required ratio finally re- 
duces to 


(B.29 (% _ ab савыға, Ө®ФабеТас Зоъссть , бо? 
) a AE ас bc taj 


The last of equations (6.28) is precisely (B.22). The means of the other 
ratios in (6.28) can be obtained in a manner similar to that which is used 
here for ab/c?. It must be remembered that in the derivation of the formu- 
las (6.28) it was assumed that a/à, 8/5, y/ē are so small, in each case, that 
Powers higher than the second can be neglected. 
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B.9. Derivation of the Principal-Factor Method 
The first stage of the principal-factor method involves the selection of the 


first-factor coefficients ал so as to make the sum of the contributions of that 
factor to the total communality a maximum. This sum is given by 


(7.35i») А = а фа +... + а, 


and the coefficients ал must be chosen so as (о make А, a maximum under 
the conditions 


(7.4%) тк = S анамы %Е-1,%,..;,%); 
t=1 


where r;; is the communality A? of variable 2). 

In order to maximize a function of n variables when the variables are 
connected by an arbitrary number of auxiliary equations, the method of 
Lagrange multipliers* is particularly well adapted. This method will be em- 
ployed to maximize Aj, which is a function of the n variables ал, under the 
n(n + 1) conditions (7.4) among all the coefficients ау. Let 


я o 
(B.23) 2T = Ai Ут = Ar — У D инанам, 
i k=1 


j,k=1 t=1 


where и, (= шь;) are the Lagrange multipliers. Then set the partial deriva- 


tive of this new function T with respect to any one of the л variables ад 
equal to zero, namely, 


ar c 
(B.24) 35; = 94— нан -0, 


and similarly put the partial derivative with respect to any of the other 
coefficients ау (t >< 1) equal to zero, that is, 


oT + 
(В.25) A" Оннан =0 @ 51). 


Тһе two sets of equations (B.24) and (В.25) may be combined as follows: 


aT : 
(B.26) Bua бал — M инам =0 (4-1,2 


= ›++#›%), 


where б, = lif ¢ = land ôu = 0 if ғ 1, 
* See, e.g., William F. Osgood, Advanced Calculus, 1932, pp. 180-85. 


m WW 
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Multiply (B.26) by aj; and sum with respect to 7, obtaining* 


n n n мі 
(В.27) b> а, - >. M алапа = 0), 
ў=1 ісі 


гі 
п 

Now, the expression X enan is equal to ам according to (B.24), and, 
гі 


т 
setting У aj, = №, equation (В.27) may be written as follows: 
=! 
j=1 


(B.28) ôrdi — X anan = 0. 
ті 


Upon multiplying (B.28) Ьу ау and summing for 2, this equation becomes 


(B.29) айм = Э = 0, 


k=1 t=1 


or, by using (7.4), 
(B.30) X raan = Мал = 0. 
к=1 
The expression (B.30) represents n equations, one for each value of j. 
Writing these equations explicitly, there results 


(Mj — Ха + туа Frutat... + тал = 0, 

Тай + (8 — Х)а — 9A, +... + 7а = 0, 

(В.31) тла + 7305) + (08 — Ма +... + 70 = 0, 
Лаба F Tala + таз +... + (М — Ayan, = 0. 


These equations furnish the basis for the solution of the unknown coeffi- 
cents ад. The remaining description of the method is presented іп 7.3. 


B.10. Direction Cosines of Oblique Reference Axes 
One stage in the process of obtaining an oblique factor solution involves 
the calculation of the direction cosines of these reference vectors with re- 


* This procedure was suggested by M. A. Girshick, “Principal Components," Journal 
of the American Statistical Association, 1936, pp. 519-28. 
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spect to the original orthogonal reference system. In 11.3 two methods were 
suggested for the determination of such direction cosines. One method is 
based upon averages of points representing the variables, and the other de- 
pends upon composite variables. It will now be shown that these two ap- 
proaches lead to identical values of the direction cosines. 

For simplicity, let there be two initial orthogonal factors, F, and F», and 
eight variables consisting of two subgroups. Let the composite variables of 
these subsets be defined by 


vu = 2 + z+ 23d 24, 
Vo = 25 + 2 + 27+ 2s. 


Then the coordinates of the point representing the first of these variables 
are given by 


4 4 
1 1 
(B32) А: (ss EF >, жы У) 
j= j= 


and a similar expression is possible for the second variable. The distance 
of this point from the origin is 


(B.33) ГОР) = Vite, + rie, 


and the direction cosines of the line from the origin through this point are 
Tur Tw. 

B.34 = а=. тсс Кы. 

( 3 ) An DOP) and № D(0Pj) 


Instead of employing the composite variable, the centroid of the first 
subgroup may be obtained, as follows: 


4 4 
1 
(B.35) Qi: (ээ 225 Р 
ј= j= 


Now it may be noted from (B.32) that 


4 


4 
2 Уту, АЛАЛЫ 5 ;lir, = 97р, , 


ігі ігі 
and hence the distance of the point 0; from the origin is 


(B.36) D(0Q:) = 1o. Vire, + тї», = іш, (ОР). 


—— 


r— 
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Then the direction cosines of the line OQ; are given by 


1 1 
aIv oF, To. F, i05, oF, 


Fa DOP) 7 DOP) 7 ™ 


(B.37) 
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ic, D(OP) — Г(ОР) 


From (B.34) and (B.37) it is evident that either the method of composite 
variables or the method of averages yields the same values of the direction 
cosines of the axis passing through the first cluster of points. Similar results 
can, of course, be obtained for the axis through the second cluster of points, 
and the method can readily be generalized to any number of variables in the 


subgroups and any number of factors. 


B.11. Matriz Formulation of the Relation between a Pattern and a Structure 


In matrix notation a factor pattern expressing the N values of each of the 
n variables in terms of the factors may be written in the form: 


(B.38) Z=MF, 
where 
Ға Fe 
2 212 Zin a ж 
7- 200 2о 2ом Bee Ға Fia 
. Un Uis 
Zn ®м Zan | "E 
Um Une 
and M is the complete pattern matrix, 
|an аю ... (вт @ 0 
M = аз 49... Am 0 а: 
бас бәк aes Gaa 0 0 


Postmultiplying both sides of (B.38) by F’, and multiplying by the scalar 


1/N, this equation becomes 


(B.39) 5 ZF'-M (5 FF’) : 
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The expression in the left-hand member of equation (B.39) can easily be 
shown to be the factor structure 


fi he tim a 0 0 
_ Ж, gue dé іт 0 аз 0 

peu qeu an RE ракы. TEM 
ha ба s б 0 0 sos Gy 


where £j, = тағу and a; = ту, according to (2.18). The expression in the 
right-hand member of (B.39) can also be simplified by means of the relation 


БЕ; ЁК ОҒ бы ҒЫ Uni 
1 wp 1 || ЁЁ: ZFii ZF miU EF miUni 
N Ууру, 20,” >Ш DU as 

ZU,F Lakai ZU.U ZU 


Since all factors are in standard form, each element in this matrix represents 
& coefficient of correlation multiplied by N. Hence, the preceding expres- 


Sion is equal to the matrix of correlations among the factors, namely, 


(B.41) vik -%- IH ч | 


where the matrix of correlations among the common factors is given by 


1 ТР,Р, „Б, TR, Fm 
% ТЕ,Р, 1 TF LF m 
Tem; Тжығы ... 1 


Now, substituting (B.40) and (B.41) into (B.39), the latter reduces to 
(В.42) 5 = МФ, 


which states that a factor structure is е 
multiplied by the matrix of intercorrelati 
lationship shows clearly that if the fact 
identity matrix) then the elements of t 


qual to the pattern matrix post- 
ons among all the factors. This re- 
ors are uncorrelated (і.е., Ф is the 
he structure are equal to the corre- 


- 0 co-———mÁ——— E oa 
аадыны Мыл... 2 


De -~E dass c 
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sponding elements of the pattern. The explicit expression for the pattern 
matrix can be obtained from (B.42) by postmultiplying both sides by $71. 
The result is 


(B.43) М -5%-:. 


Sometimes it is convenient to have the relationship between a pattern 
and structure for the common-factor portion only. If f is used to represent 
the matrix of common factors, А the matrix of common-factor coefficients, 
and T the common-factor portion of the structure, then by analysis similar 
to the preceding it can be shown that 


(B.44) T = Аф. 


This relationship between the common-factor portions of the pattern and 
Structure is analogous to that for the complete solution. 


APPENDIX С 
OUTLINE FOR COMPUTATION OF А BI-FACTOR PATTERN 


А bi-factor solution can be obtained from a given matrix of correlations 
by the following calculations. This is the method developed in 6.4 and ге- 
duced to a routine procedure. For concreteness the method will be de- 
scribed in relation to the computation of the pattern for the data of 2.8. 
The detailed procedure is presented as a series of steps.* 

1. Determine the grouping of variables by the method of B-coefficients, 
as developed in 2.7, for the given matrix of correlations. The method of 
computing B-coefficients is completely exhibited in 2.9 and will not be re- 
peated here. For the given data, the B-coefficients are computed in Table 
2.3, and the groups of variables indicated in (2.35). 

2. Arrange the matrix of correlations so that the variables constituting 
a group are together. Of course, only the correlations below the principal 
diagonal need be written because the matrix is symmetric. The variables 
in Table 2.2 have been arranged according to their respective groups. It 
may be convenient, but not at all necessary, to renumber the variables at 
this stage from 1 to n. In the present example the variables are already 
numbered consecutively from 1 to 24 when they are arranged by groups. 

3. As a result of the B-coefficient analysis, there are postulated six fac- 
tors, viz., the general factor and five group factors, one for each group of 
variables. This hypothesis may be changed if the ensuing analysis indicates 
necessary modifications. 

4. In the following analysis it is assumed that the matrix of correlations 
contains only essentially positive entries. This condition has always been 
found ‘to be met with psychological data; but high negative correlations 
might occur for different variables. If there are one or two negative corre- 
lations which are insignificantly different from zero, they may be set equal 
to zero for purposes of computing the general-factor coefficients. On the 
other hand, when an observed matrix includes significantly high negative 
correlations, the variables producing these values may be changed in direc- 
tion in an attempt to reduce the matrix to essentially positive correlations. 
In the example there is one negative correlation, namely, 13,19 = —.075, 
which is insignificant, being only nine-tenths of its standard error. Hence 
a value of zero is used for this correlation in the following step. 

Б. To obtain the general-factor coefficients, formula (6.14) is used. The 
component parts of this formula will be calculated piecemeal, however, to 
clarify the procedure. First, add all the correlations of any variable s (seG;) 

* The essential steps in this outline are hased upon those of Miss Frances Swineford 
given in Student Manual of Factor Analysis 
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with each variable j of a group Gu (u #7, u = 1, 2,..., 5). This sum 
may be written 
à 


JG, 
where s is a fixed variable in G;, and Gu is any other group. The number of 
such sums for n variables and m groups is 


PC = р) = mn —n = n(m — 1), 

ізі 
where р; is the number of variables in the group G;. In the example there 
are 24(5 — 1) — 96 such sums. These sums are obtained from Table 2.2 
and are presented in Table C.l. Looking down the С, column, the first 
entry is 

Уы = ты + ть + Тыз + rss = 1.080. 

жа 
Similarly, any other entry represents the sum of the correlations of the 
variable in the stub with all the variables in the group indicated at the head 


of the column. 


TABLE C.1 
SUMS OF CORRELATIONS FOR VARIABLES WITH GROUPS 
Gi Gi Gs с Gs 
1.... 1.591 1.903 
G2] .878 1.380 
n 3... 1.205 1.298 
4 1.090 1.651 
Bus 1.357 2.012 
Був 1.525 1.948 
С 7... 1.201 2.003 
Bes 1.502 2.035 
9... 1.506 2.123 
10.. 1.246 1.531 
glll 1.906 1.568 
gii MM 1.160 1.558 
18... 1.395 1.685 
(14.. 
I5. 
үү M 
бүт... 
18.. 
19.. 
20. 1.293 2.157 .862 
21....| 1.189 1.523 1.584 
G4922....| 1.275 2.063 .936 
28.. 1.540 2.276 1.245 
24.. ‚944 2.102 1.715 
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6. Using the values below the diagonal of Table C.1, compute the fol- 


lowing intergroup sums: 
> У Taj; 
*G; 4G, 


that is, the sums of all the correlations of the variables of one group with 
those of any other group. There are evidently (т) = m(m — 1)/2 such 


2 


sums for m groups. For the example the ten sums appear in Table C.2. The 
first entry is 


>>») = 1.080 + 1.164 + 1.019 + 1.262 + 1.030 = 5.555, 


веб \ jeGi 


while any of the others is similarly obtained. The sum in the parentheses 
was obtained for each s, in step 5, and now the summation is taken for 8 
varying over a specified group. Тһе values above the diagonal in Table C.1 
should be used to check the entries of Table C.2, but the sums need not be 


written twice in the latter table. For example, the first entry in Table C.2 
could also be obtained as follows: 


У(Х) = 1.618 + 1.028 + 1.304 + 1.605 = 5.555. 
ath ro 
Thus a check on all the additions is provided. 


TABLE C.2 


SUMS ОЕ INTERGROUP CORRELATIONS AND DENOMINATORS 
FOR GENERAL-FACTOR COEFFICIENTS 


Total...| 19.626 28.039 20.387 


26.138 31.280 | 25 =125.470 


, у= 62.735 
Denominator| 43.109 34.696 42,348 86.597 81.455 as aise oo 


7. It will be noted from (6.14) that the d 
of the general-factor coefficient for variable z, is the sum of all intergroup 


correlations exclusive of the group which includes z,. The denominators 
for all variables of a group are therefore alike, 


enominator for the calculation 


--- 


Ў 


a a 
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8. Add the entries in the row and column corresponding to each group 
in Table C.2 and record the sum in the "Total" row. Thus, for the group 
G4, say, the Total is obtained as follows: 


4.764 + 7.091 + 5.707 + 8.576 = 26.188 . 


9. The sum of the “Total” row should then equal twice the sum, S, of 
the entries in Table C.2. In the example the sum of the entries is S — 
62.735, and the sum of the “Total” row is 125.470, or 28, completing the 
check. 

10. Subtract from S each Total in turn, entering the difference in the 
“Denominator” row. For example, the denominator for the calculation of 
the general-factor coefficient for any variable in Gi is 


62.735 — 19.626 = 43.109. 


11. There is another computational check available at this point. Each 
Denominator is obtained by subtracting a Total from 8. Hence, if there 
are m groups, the sum of the “Denominator” row is equal to mS minus the 
sum of the “Total” row. Since the sum of the “Total” row is always twice 
the sum of the entries in the table, it follows that the sum оѓ the “Denomi- 
nator" row must Бе (m — 2)S. In the example, the sum of the *Denomina- 
tor" row is 188.205, which checks with (5 — 2)5. 

12. Tables C.1 and C.2 may now be employed to get the general-factor 
(B) coefficients. Each aq, or 2,5, 18 а fraction whose numerator is the sum 


of the М 2 Э = (2) = 6 products by pairs of the m — 1 = 5 — 1-4 
entries in row s of Table С.1, and whose denominator is the Denominator 
in the column of Table C.2 for that group which includes variable zs. 

13. The calculation of the numerator of a2), when there are five or more 
groups, may be shortened with the aid of an additional table. It will be 
noted that the first entry in a given row of Table C.1 is to be multiplied by 
each of the remaining entries in turn. Construct Table C.3 so that the first 
entry in any row is the sum of the m — 2 = 3 entries beyond the first entry 
of the corresponding row of Table C.1. Similarly, the second entry in а 
row of Table С.З is the sum of the m — 3 = 2 remaining entries in the 
corresponding row of Table C.1. The other entries of Table C.3 are ob- 
tained in like manner, the final value for each row being the same as that 
in Table C.1. The total number of entries in each row of Table C.3 is one 
less than that in Table C.1. | 

14. The columns of Table С.З are во headed, and the entries recorded, 
that the sum of the products of the entries for variable z, in Table C.1 by 
the corresponding entries in Table C.3 gives the numerator of а?,. Table C.3 
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may then be folded for each row and placed over Table C.1 so that corre- 
sponding entries are directly under one another. The sums of the products 
of these corresponding entries give the desired numerators. 


TABLE C.3 
FoR THE CALCULATION OF THE NUMERATORS FOR 
GENERAL-FACTOR COEFFICIENTS 


Gi Ga ба Gi 
зар E nera 4.721 3.494 1.903 
G) 2 е aaah orc 2 2.858 2.267 1.389 
MY Bonon] pine. reete 3.055 2.503 1.298 
Гг КӨРЕР Р 3.437 2.741 1.651 
Drains въ 4.583 3.369 2.012 
[MET 4.433 3.473 1.948 
[t p PM 4.208 3.204 2.003 
y M 4.788 3.537 2.035 
9,4“. 4.472 3.629 2.123 
3.992 2.777 
4.983 3.474 
3.593 2.718 
4.753 3.080 
14 3.442 2.122 
15.. 2.952 1.877 
G,19. 3.675 2.479 
NUT ТТ? 3.682 2.444 
18.. 4.038 2.991 
119.. 3.585 2.370 
720. 4.711 2.554 1.692 
191 4.731 3.208 1.026 %,.... 
6,22. 4.708 2.640 1.704 
123. 5.241 2.965 ЖӘН аараан 
24. 5.653 | 3.551 15880 рангаи 


15. When working on a calculating machine, these numerators need not 
be copied down but can be directly divided by the denominators, which are 
given in Table C.2, to get the squares of the coefficients. For example, 


1.605(3.437) + .696(2.741) + 1.090(1. 
а% t a6 1+ 10900.65) _ 21396. 


Record the squares of the coefficients in Table С.4. 


16. Without the short cuts described in this outline, each a2, of the ex- 
ample if computed directly by formula (6.14) would involve 121, 134, or 149 
products in the numerator and the sum of an equal number of correlations 
in the denominator, depending on whether the variable z, is included іп a 
group of 6, 5, or 4 variables, respectively. The method described in step 12 
reduces the large number of products for each numerator to 6, while the 


—— 
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method of step 13 reduces this number to 3. Of course, а certain amount of 
time is required in obtaining the sums of Tables C.1 and C.3, but the eventu- 
al saving of time is tremendous for a large number of groups. 

17. The general-factor coefficients are the square roots of the values cal- 
culated in step 15. These coefficients also appear in Table C.4. In practice, 
these coefficients need not be written down in this table but can be put di- 
rectly in the final factor pattern shown in Table 6.8. 


TABLE C.4 
GENERAL-FACTOR COEFFICIENTS 


н а% азо Шы aly аф 
34687 589 181... usn 33162 576 
12752 357 i эге 4 15036 388 
16074 401 1. ЖИНИ 12298 351 
21396 463 РРР 24622 496 
33923 582 Д а чаа 17804 422 
33044 575 ТЕТІ 26573 515 
28563 534 1: oue n 19544 442 
38978 624 || 20...... 41516 644 
31308 560 || 21...... 41594 645 
15036 388 22.5 & 41448 644 

.27191 .521 28... „+ .53921 ‚734 
.16323 .404 84... ues .50705 .712 


18. Now the general-factor residuals must be obtained, and then the 
group-factor coefficients can be calculated. To facilitate the calculation of 
these residuals, a table of products of general-factor coefficients may be pre- 
pared. This table of а;оако is presented in Table С.5 for the given data. А 
convenient procedure for the calculation of the elements of Table C.5 is to 
proceed by columns, as follows: multiply аш = .589 by every other ако 
(k = 2,3,..., 24), in turn, to obtain the first column of values of Table 
C.5; then multiply азо = .357 by all remaining aio (k = 3, 4,..., 24) to 
get the second column of values of Table C.5; and so forth for all remain- 
ing columns. 

19. Subtract the values in Table C.5 from the corresponding entries of 
Table 2.2 to get the general-factor residuals. Record these residuals below 
the principal diagonal in Table C.6 (the meaning of the values above the 
principal diagonal will be clarified in step 28). Thus the general-factor resid- 
ual for variables 1 and 2 is 
— аңа = -318 — .210 = .108. 


Table 2.2 while the product 010020 = 
‘erence .108 is the residual #12 which 


Ta = Тю 


The correlation тә = -318 comes from 
:210 comes from Table C.5, and the diff 
is entered in Table C.6. 


-leze leer" leer 169 lere 11981008 jese’ 1082 |923" 01 88 |128 912 668 РРР’ 088 60? РТР |066” 
У 8601682 |ecr |760 1286 882 11+ |80 |066 (00Р |427 ОРЕ 
922 [OSZ |12809 |988 |02 |198 (ZOP ` РРБ 0/6 |026 |860” 
922109 [ZLE |192 |988 |090 |1987 с0Р` РРБ [TLE |946 |660” 
922 [OSZ |12092 |968 |022 |196 СОР РР 016 |2767 |860” 
еет”|12171662|6217|082 [ILT |8Р2 |922 |962 |20 |150 |900” 
181 |002 |766 |802 |890 |002 |886 |126 |612 |062 |006 |860” 
SFI [9T Eh] 104171022 |9917 |982 |692 |600 |60 |990 |261” 
%/17|061:|982 1008 |852 |2617 |820 |016 |690 |280 |680 |060” 
7771981 [206 (с |881 |961 |46 T |612: |281 200 |706 |691” 
р вес 16121006 |161 012 сс |202 |906 |900 |081” 
рес |006 |600 |626 |628 |806 |186 |066 |790” 
Р “** OTS AST |92262 |918 (сс |666 11817 
қ 52221006060 |906 |872 008 |606 106 
"v М рте ева |2006 |600 1000 |081” 
ке | ү ` '''|ere' 1662226926 |690” 
a sn 7771226688696 |680” 
p к д КИ БИБЛИЯ ХА 
9 777 666992 
ee m > кз Т. боб: 
sz | ee | zz | iz} os| or] ar] zt | or | er | sr jet | er | it [OL ув | es} 2]9 | 8 |+ 1 


opp :SLNAIOIAAOQ YOLOVY-IVUANGY ЛО SLOngaouq 
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20. А check on the subtractions involved in the calculations of the residu- 
als is provided by means of the table of products. The general-factor re- 
sidual for any two variables z; and 2; is given by 


Tj = Тук — адан, 
and the sum of all such residuals for j fixed and all k > jis 


n n n 
>, Tuc > Tik - 2. айко. 
k=j+1 k=j+1 кеі 
In other words, the sum of the entries below the principal diagonal of any 
column, j, in Table C.6 is equal to the difference between the corresponding 
sums in Tables 2.2 and C.5. Record these sums in Table C.7 and check that 
the sums obtained from Table C.6 are actually the required differences of 
sums from the other tables. 


TABLE C.7 
CHECK ON CALCULATION OF RESIDUALS 


21. If the grouping of variables is reasonable, the general-factor residual 
correlations of Table C.6 will all tend to be insignificant except those within 
the groups. Thus for Gi, the intercorrelations are consistently positive and 
in general greater in value than the remaining correlations with the бі 
variables, while the residual correlations of the variables in G, with the 
others are all insignificant. The same is true for б» and G,. The group Gs 
also has this property with one exception—the value 71,24 = .255 is sig- 
nificantly different from zero (shown іп 6.10). The group бу, however con- 
tains a number of negative residual correlations and very small positive 
ones, clearly indicating that no additional factor is required for these vari- 
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ables. The general-factor residuals among the variables of G; are insignifi- 
cant and may be considered as final residuals. 

If the analysis of the table of general factor residuals indicates numerous 
revisions of the original plan, it may be expected that the general-factor 
coefficients will change somewhat. A more accurate solution may then be 
obtained by recalculating the general-factor coefficients from the revised 
plan. In the present example the modifications are so slight that such a pro- 
cedure is not deemed necessary. 

22. To obtain the group-factor coefficients, formula (6.16) is used. The 
coefficients of B, are calculated in Table C.8, where all the triads are put 


TABLE C.8 
CALCULATION OF THE COEFFICIENTS OF B, 


Test 1 Test 2 Тевт 3 | Test 4 

Jk tjk ЈК lik jk tjk jk tjk 
28) ы усе 1087. (| 19. sso as 1125 22601 || 12» s so ‚1174 
S рев кате 3240 || 14........ 0360 1019: || 38... sus ens .1390 
BUS с зе ak 2737 | 34 0950 SEE C Fm .0445 
[i MN .2338 || a3,........| -0812 | аї,........| .2290 | аң........ .1003 
LET 7:7 Sas ox .| -285 p 49 ao e өсер ‚317 

| 


down explicitly, the average obtained in each case, and the square root ex- 
tracted. The number of variables in G, is рі = 4, во that there are just 


: = 
(2) = 3 triads for the calculation of each coefficient. Thus, the first coeffi- 


cient is obtained as follows: 


ай, = 10 + tes + ім) 


1/408 X 167 | 108 х 195 , 167 X .195 
3 .174 .065 119 


= 3(.1037 + .3240 + .2737) 


.7014 
= 13 = 2338, 


and 
ay = У.2338 = 484. 
ed in а similar manner in 


23. The coefficients of В, and Вз are comput 
coefficients, there are 


Tables C.9 and C.10. In the calculation of the В; 


P - — 6 triads in each case, whereas for each of the Вз coefficients 
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there are again only 3 triads since there are 4 variables in G3. The signifi- 
cant general-factor residual between Tests 10 and 24 will be treated in step 26. 


TABLE C.9 
CALCULATION OF THE COEFFICIENTS OF B, 
Test 5 Test 6 Test 7 Test 8 Test 9 
jk tjk jk tjik jk tjk jk ljk jk tik 
EI sic ees .2386|| 57..... .9452 1259 | 56..... .5422 
88... s .8673|| 58..... .2243 1782) 57..... .4442 
89. 5 os .2907 || 59..... .2834 2090 | 58..... .3379 
Өз css Sue .2694| 78..... .2438 1158 67..... ‚3646 
МӨ; алда 8548) 79..... К 4215) 07841 68..... . 4270 
BO ажо» .4664|| 89..... -3599| 1356|| 78..... .2470 
l TIE -8295)| аф..... .3130, 1405] а?,..... .3938 
UU TRE -574 || an..... .959 375 || ау..... .628 


TABLE C.10 
CALCULATION OF THE COEFFICIENTS ОЕ В, 
Test 10 Test 11 Test 12 Test 13 
jk tik jk tjk jk ljk jk tik 
UE Is eos 5587 | 10, 12..... 1436 || 10, 11..... 8809 || 10, 11..... 1542 
nen C P 2220 || 10, 13..... 3582 || 10, 13..... 6455 || 10, 12..... 1206 
IIS лемін 2838 | 12, 13..... 1836 || 11,13..... 2588 | 11, 12..... 3008 
tissima xs « 3532 | at, 2285 || at, , 4117 | аһ,...... 1918 
ба» eres 594 [^ M PI 478 бар 22 642 GUESS мел 8 


24. It will be noted that the values of the triads for the calculation of any 
one of the preceding group-factor coefficients are fairly constant, i.e., they 
satisfy (statistically) the conditions for one common factor. If the triads 
are written out for the calculation of ay, from all the tests of G4, it will be 
found that they vary from 


= Tis, fio ia t .001 x .035 
Tas .130 = 0003 


Йв,19 


to 


з, мв, м _ .234 x 220 
hiis = D 151 = 3409, 
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with five triads nearly zero and the other five significantly different. Clearly, 
the six variables of G4 do not measure a single factor, and the original pat- 


tern plan must be modified for these tests. 
25. By various procedures in 6.10, it is shown that the revised plan may 
be formulated as follows: Tests 14 to 17 involve a single factor, say B4, and 


TABLE C.11 
CALCULATION OF THE COEFFICIENTS OF B, 


Test 14 Test 15 Test 16 Test 17 
jk tjk jk tjk jk tik jk tik 
15, Bross 3409 || 14,16..... 1606 || 14,15.....| .1420 || 14,15..... ‚1490 
DOS: v 5 5 2102 || 14, 17.. 2604 || 14,17..... 1429 || 14,16..... .0925 
16, 11575; 3386 || 16, 17..... 2587 || 15, 17..... 0881 || 15, 16..... .1500 
6 gies. nexa 2966 || аф... .2266 || ai. .1248 || at... .1305 
ац,4 545 а, .476 ITE ovde ais .353 апа...) -361 


Tests 17 to 19 involve another factor, say Bs, so that Test 17 is of complexity 
3 while the other tests are still of complexity 2. The coefficients of B, are 
then calculated in Table C.11. Since there are just three tests that measure 
Bs, each of their coefficients is given by one triad. Thus 


NER = М2434 = 493, 


ать = ТЕ 


к= 231 1180 
Qi85 = Vlr. = XD = V.2192 = .468, 


— [137 x .180 езе 
а, = Visas = = аша 0771 = .278. 


26. Another modification of the original pattern plan involves the sig- 
nificant general-factor residual 719,24 = .255 between two variables which 
were not together in the initial grouping. A doublet is postulated for the 
two tests, and its coefficients, according to (6.50), are taken to be 


di = da = V.255 — 117 = 4/188 = .371. 


the doublet coefficients, and the general- 


27. The group-factor coefficients, 
d in Table 6.8 to give the final factor 


factor weights may now be entere 
pattern. 


28. The final residuals can now be obtained by means of the formula 


Ға = fj — Оки + 
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If the variables z; and 2, do not measure the same group factor, then their 
general-factor residual is the final residual. On the other hand, if variables 
j and k involve the same group factor, then the product of their group-factor 
coefficients must be subtracted from their general-factor residual to obtain 
the final residual. For example, Tests 1 and 5 are in different groups, and, 
therefore, their general-factor residual, — .022, is the final residual, Tests 
6 and 9, however, involve the same group factor, and their final residual is 
not .392 but 


7.9 = .392 — .559(.628) = .041. 


Record the final residuals for the variables that involve a common factor, 
beyond the general, above the principal diagonal of Table C.6. 


TABLE C.12 
FREQUENCY DISTRIBUTION OF FINAL RESIDUALS 


Value of Residual Frequency Value of Residual Frequency 


© 
ЮКЕ qi 


17 Standard 4еуїайоп.... 0655 


29. According to the methods of Chapter VI, the factor pattern of Table 
6.8 is а valid description of the twenty-four psychological variables, and the 
factorization has been carried to the proper stage as judged by the various 
statistical tests. The degree of factorization represented by this pattern is 
also in agreement with the more crude criteria for "when to stop factoring" 
which have hitherto been followed in factorial analyses, namely, that the 
communality of a variable should not exceed its reliability and that the 
standard deviation of the final residuals should be of just about the same 
magnitude as the standard error of a zero correlation in a sample of equal 
size. In Table 6.11 it is shown that the first of these standards is met, while 
Table C.12 gives the frequency distribution of the final residuals of Table 
С.6. Since the standard deviation (.0655) of the final residuals is less than 
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the standard error (.0830) of a zero correlation, the second standard is 
also satisfied. 

30. The variance of each variable which is not accounted for by the 
common factors can easily be deduced by the formulas of 2.3. In Table 6.11 
the communality and uniqueness of each variable is presented, and then 
the coefficients of the unique factors are obtained. These are given in Table 
6.8 along with the common-factor coefficients. Each row of this factor pat- 
tern then completely expresses a variable as a linear function of the factors. 


APPENDIX D 


OUTLINE FOR COMPUTATION OF A 
PRINCIPAL-FACTOR PATTERN 


A principal-factor pattern can be procured for any matrix of correlations 
by the iterative method of 7.4. 'The form of the numerical calculations is 
exhibited in the following steps, in which the method is applied to the ex- 
ample of eight physical variables introduced in 7.5. 

1. Determine the best possible estimates of the communalities for the 
given set of variables. This problem is considered in detail in 4.6 and in 7.2, 
and will not be repeated here. For the illustrative example, the choice of 
communalities is discussed in 7.5. 

2. Insert the estimates of the communalities in the principal diagonal of 
the correlation matrix and fill in the complete matrix. It has usually been 
suggested that only the lower-left half of a matrix of correlations be written, 
Since such a matrix is symmetric, and the correlations of any variable with 
all the others can be determined from this portion. In the present method 
of analysis, however, it will be found more convenient to write the sym- 
metrie matrix in full in order to facilitate the squaring of the matrix. The 
correlation matrix R for the given data is presented in Table D.1. 


TABLE D.1 
CORRELATION MATRIX: R 

1 2 3 4 5 6 7 8 5; а) 
1 854 .846 .805 .859 .473 .398 .301 .382 4.918 1.000 
2 846 .897 .881 .826 .376 .326 .977 .415 4.844  .985 
3 805 .881 .833 .801 .380 .319 .237 .345 4.601 .936 
4 859 .826 .801 .783 .436 .329 .327 365 4.726 .961 
5 473 .376 .380 .436 .870 .762 .730 .629 4.656 .947 
6 398 .326 .319 .329 .762 .687 .583 .577 3.981 .809 
7 801 .277 .237 .327 .730 .583 .521 .539 3.515  .715 P 
8 882 .415 .345 .365 .629 .577 .539 .579 3.831 .779 


3. Obtain the sums of the rows (or columns 


) of R. For any variable j 
this sum is 


n 


Sj > у Tik, 
k=1 


in which the estimate of the communality h is used for туу. For the ex- 


ample, n = 8 and the sums are given in the column alongside the matrix 
of correlations in Table D.1. 
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4. To determine the first-factor coefficients, take as the set of trial values 
the numbers af}, which are the quotients of S; by the largest (absolute) 
sum. It should be noted that these numbers will not be used immediately 
in equations like (7.12), but it is convenient to refer to them as trial values. 
The numbers af} (j = 1,2, .. ., 8) for the example, which are obtained by 
dividing the numbers S; by 4.918, are also given in Table D.1. It may be 
found more convenient, especially if the number of variables is great, to 
obtain the reciprocal of the largest S; and to multiply all the sums by it. 

5. Square the matrix of correlations. Any element, off the diagonal, of 
the resulting matrix R? is the sum of all the products of corresponding ele- 
ments in two columns of R. The diagonal elements are merely the sums of 
the squares of the elements in the corresponding columns of R. Thus, the 
element in the jth row and the kth column of В? is given by 


утыгы (Е = 1, 9, sog Ma 


т-1 


The matrix В? for the given data is presented in Table D.2. The entry in 


TABLE D.2 
SQUARE OF CORRELATION MATRIX: R? 
1 2 3 4 5 6 7 8 s? mun at 
1 || 3.450 3.450 3.301 3.325 2.577 2.185 1.903 2.179 || 22.370 22.370 1.0000 
2 || 3.450 3.475 3.322 3.333 2.471 2.093 1.815 2.115 22.074 22.075 .9868 
3 | 3.301 3.322 3.181 3.188 2.341 1.983 1.718 2.003 || 21.037 21.039 .9405 
4 || 3.325 3.333 3.188 3.213 2.461 2.084 1.810 2.085 || 21.499 21.499 .9611 
5 || 2.577 2.471 2.341 2.461 2.966 2.550 2.278 2.372 20.016 20.016 .8948 
6 | 2.185 2.093 1.983 2.084 2.550 2.200 1.965 2.041 17.101 17.102 .7645 
7 || 1.903 1.815 1.718 1.810 2.978 1.965 1.765 1.820 15.074 15.074 .6799 
8 || 2.179 2.115 2.003 2.085 2.372 2.041 1.820 1.925 16.540 16.540 .7394 


the second row and third column is given by 


8 


Б жайа = .846(.805) + .897(.881) + .881(.833) + .826(.801) 


vel 
+ .376(.380) + .326(.319) + .277(.237) + .415(.845) = 3.322, 


while any of the others is obtained similarly. Of course, only the diagonal 


elements and the entries above (or below) the diagonal need to be calculated 
because the square of a symmetric matrix is also symmetric. The complete 


matrix is written, however, for convenience of further squaring. 
6. A check is available on the squaring process. Compute the product of 
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R by the column of values S; of Table D.1. The result is the column of 
values T? of Table D.2. These values are given by the following formula: 


8 
TP = S тыб, G21,2,...,8), 
т=1 
so that the first entry, for example, is 
8 
Tj? = У? raS = .854(4.918) + .846(4.844) + ...--.382(3.831) = 22.370. 
2-1 


Now, add the rows of R? and denote the respective sums Бу 82, Corre- 
sponding values of Sí? and Tí? should agree except for errors of rounding. 

T. Take as the next set of trial values the numbers af}, which are the 
quotients of T? by the largest check sum T(?. Every number T? is di- 
vided by 22.370 in Table D.2 in obtaining the trial values а). 


TABLE D.3 
FOURTH POWER OF CORRELATION MATRIX: R* 


1 2 3 E 5 6 4 8 5% m alt 


ji 
65.54 64.94 61.95 63.06 56.04 47.80 42.00 46.59 | 447.92 447.93 1.0000 
64.94 64.38 61.41 62.49 55.27 47.13 41.41 45.98 | 443.01 443.02 .9890 
61.95 61.41 58.59 59.61 52.67 44.91 39.45 43.82 | 422.41 422.43 .9431 
63.06 62.49 59.61 60.67 53.86 45.93 40.36 44.78 | 430.76 430.76 .9617 
56.04 55.27 52.67 53.86 50.40 43.06 37.97 41.61 | 390.88 390.90 .8727 
47.80 47.13 44.91 45.93 43.06 36.80 32.45 35.55 | 333.63 333.64 .7448 
42.00 41.41 39.45 40.36 37.97 32.45 28.62 31.33 | 293.59 293.60 .6555 


46.59 45.98 43.82 44.78 41.61 85.55 31.33 34.39 | 324.05 324.05 .7234 


ONTA OO tO — 


8. Compare the trial values af? with the corresponding af? of the first 
set. When there is no appreciable variation between two successive sets of 
trial values, then the squaring process has been carried far enough. This 
is the scheme for determining the number of times а matrix should be 
squared. In the present case the values а(2) differ sufficiently from the corre- 
sponding af} (see Table D.5) so that the need for further squaring is dis- 
closed. The standard recommended is to obtain agreement between succes- 
sive trial values to within five units in the last decimal place desired in the 
final results. Here it is planned to retain three decimal places for the factor 
coefficients, the same as in the original correlations. Hence the absolute 
values of the discrepancies between trial values should be less than .005. 

9. Square the matrix of correlations a sufficient number of times to make 
the successive sets of trial values approximately equal. In calculating the 
various powers of R, always determine the check column Т; first. Thus for 


ЭНЕ 
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the eth power of R, the column of values T(? can be calculated from Ве and 
842 before the elements of Ве are obtained. Then the values of aff may 
be computed and if they agree with the values aft/?, the calculation of the 
elements of R* is obviated. 

In the illustrative example, the values Tí? are obtained from В? and 
59, and the af} are calculated. These values are not close enough to the 
corresponding af? and so the elements of Rt are computed in Table D.3. 
Next, the values of 7/9 are determined in Table D.4. Since the maximum 


TABLE D.4 
EIGHTH POWER OF CORRELATION MATRIX: R$ 
1 2 3 4 5 6 7 8 59 que D 
ШЕКТІ 176738 1.0000 
Әл ЕРЕН .. 174853 .9893 
КІ 2. 166733 .9434 
КЕГІ .. 169980 .9618 
Б nucon . 153733 .8698 
D MI eas 131201 .7423 
ЖАН аа . 115430 .6531 
MIN assaw or көзге wikis ares еген ШАРЫН ОЙҒА ЕЕ Жаа 127505 .7214 


difference between al and af? is only three thousandths, this agreement is 
sufficiently close. It is therefore not necessary to caleulate the entries of 
К°. The differences between successive a determinations are summarized 


in Table D.5. 
TABLE D.5 


DIFFERENCES BETWEEN SUCCESSIVE 


TRIAL VALUES 


ku m E Е. a “0 кы Ed a? = ай) 
-000 -0000 -0000 
.002 .0022 .0003 
.005 .0026 .0003 
.000 -0006 .0001 
—.052 —.0221 — .0029 
—.044 —.0197 — .0025 
—.041 — .0190 — .0024 
—.040 —.0160 — .0020 


10. Employ the last set of trial values as the arbitrary numbers aj in 


the equation 


(7.12bis) 


n 

+ — . 

aj = > тукада j 
ізі 
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that is, multiply the correlation matrix by the trial values to obtain the 
new set of numbers ад. Then divide every number oj, by the largest one 
of these to get the next set of trial values. For simplicity of notation, again 
represent these new values by aj in the tabular arrangement. Multiply R 
by these new values to obtain the new aj, and aj; and continue this process 
until successive values of ад are practically constant. When the squaring 
of R has been carried to the stage indicated in step 9, the values of ад usual- 
ly will satisfy (7.13) immediately: 

(7.1351) ар = Хал, 


і 


that is, the values aj, will be directly proportional to the corresponding 
values aj. 


In Table D.6 the first column of arbitrary numbers aj; are the values 
449, determined by the squaring process in step 9. These numbers are then 


TABLE D.6 
CALCULATION OF THE F COEFFICIENTS 
Xue а ай ай am алУ%м/ Уха 
1.0000 4.4556 1.0000 .858 
.9893 4.4083 ‚9894 ‚849 
‚9484 4.2038 ‚9435 ‚810 
‚9618 4.2852 9618 .825 
.8698 3.8757 .8698 ‚747 
‚7428 3.3076 ‚7423 ‚637 
‚6581 2.9099 6581 ‚561 
‚7214 3.2142 ‚7214 ‚619 


в 
м = 4.4556, Уа = 6.0487 


V/V Жа = .85827 


multiplied by R to get а). For example, 


af, = .854(1.0000) + .846(.9893) + ...--.382(7214) = 4.4556. 


After all the values a/, have been obtained, they are divided by the largest 
of them (aj, = 4.4556) іп the calculation of the new quantities aj. Since 
the maximum discrepancy between the old and the new values ад is only 
.0001, these numbers are accepted as stationary. 
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11. The value of aj, corresponding to an = 1.0000 is the first characteris- 
tic root №. Then the coefficients of the first factor can be calculated by 
means of 


ал УА 
Voi, +03, +... + а, 


(7,15%) ад ғ 1, Bess өз Ы 


In the example, № = 4.4556 and the coefficients aj are given in the last 
column of Table D.6. 

19. A check on the final determination of the aj is provided by the 
fact that 


n 


Уа = №. 


ј=1 


In other words, ће sum of the contributions of the first factor to the total 
communality must be equal to the first characteristic root. The value of 
X from the analysis is 4.4556 and the sum of the squares of the coefficients 
is 4.455, so that the check is satisfied (within rounding errors). 


TABLE D.7 
PRODUCT MATRIX: Ф. = алак || 

1 2 3 4 5 6 7 8 Еһ аһ 
1 786 .... T 5.067 5.067 
2 .728 .721 5.014 5.014 
3 695 .688 .656 4% 4,783 4.784 
4 708 .700 .668 ӨЗІ 5. ++ ӛсе іше 4.878 4.872 
5 1641 .634 .605 21616 2558 .... оз: 9 4.411 4.412 
6 "547 2541 .516 .526 .476 406 Lus ee 3.763 3.762 
7 7481 .476 .454 .463 .419 .957 .315 .... 3.312 3.313 
8 | 7541 2596 .501 .511 .462 “204 .347 .383 | 3.655 8.656 


13. Before additional factors can be determined, the first-factor residu- 
als are required. А table of products of the first-factor coefficients may be 
prepared to facilitate the calculation of these residuals. The elements 
1Dix (=1рь) of the product matrix Qi = зрак] = llananl| are obtained as 
follows: the values in the first column are the respective products of au 
by itself and every other ал (j = 2, 3,---> n), the values in the diagonal 
and below it in the second column are the respective products of da by it- 
self and all remaining coefficients ал (j = 3, 4,.... п), and so on for the 
values in and below the diagonal in all the other columns. Since the product, 
matrix is symmetric, it is not necessary to write the values above the diago- 
nal. The matrix Q; for the given data is presented in Table D.7. 
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'To assist the reader, a summary of the notation referring to the various 
matrices employed in the principal-factor method is presented in Table D.8. 


TABLE D.8 
NOTATION 
Original First- First- Second- Second- 
Correla- Factor Factor Factor Factor 
tions Products Residuals Products Residuals 
BIemenfiz, s iu ads re tn Tjk Pik iTjk 2Dik a jk 
МЕИ...» к» ен да куа R Qi R 0: в. 
Sum of elements in row j of 
XB UTERIS к.» +» к-н» yosin S; Ej hn Ej Sj 
The eth power of matrix. . Re d Ri 0: R; 
Sum of elements in row j of 
eth power of matrix... .. Spo Ew 50 EY So 


14. To check the calculation of the elements of the product matrix Qi, 
first obtain the sums of the complete rows (or columns), that is, 


Ej = Dajan (1-1,2,..., т), 
kal 


and compare with the corresponding values of aj:D, where 


The sum of the first-factor coefficients for the given data is Dı = 5.906. 


Then the check, say, for the elements of the third row and column is given 
by the agreement of 


Ез = .695 + .688 + .656 + .668 + .605 + .516 + .454 + .501 = 4.783 
and 
аз, = .810(5.906) = 4.784 , 


and the remaining elements of Q; are similarly checked. The values Ёл and 
алл. are also recorded in Table D.7. 


15. Subtract the values in О, from the corresponding entries in В to get 


APPENDIXES 349 


the matrix of first-factor residuals В. This matrix is presented in Table 
D.9, and is written out in full to simplify later multiplications with it. The 
sums of the rows, Sa, are given in a column alongside the matrix Ri. These 


TABLE D.9 
MATRIX OF FIRST-FACTOR RESIDUALS: Ri 


a 


1 2 3 4 5 6 7 8 ба E 
1l 2118 418 .110 2151 —.168 —.149 —.180 —.149 —.149 — .6082 
2| .118 .176 .193 .126 —.258 —.215 —.199 —.111 —.170 — .6939 
3| .110 .193 «147 133 —.225 —.197 —.217 -.156 -.182 — .7429 
4 2151 .126 .133 „102 —.180 —.197 —.136 —.146 —.147 — .6000 
5 | —.168 —.258 —.225 —.180 .312 .286 ‚311 .167 .245 1.0000 
6| —.149 —.215 —.197 —.197 .286 .281 .226 .183 | .218 .8898 
7| —.180 —.199 —.217 —.136 З .226 .206 .192 | .203 .8286 
8 | —.119 —.111 —.156 —.146 .167 .183 .192 .196 | .176 .7184 


sums should be equal to the differences between the sums of the respective 
rows of R and Qu, that is, 


Sa = 8; — En. 


16. To obtain the best set of trial values for the calculation of the second- 
factor coefficients, an appropriate power of R, should be employed. It is not 
necessary, however, to perform repeated squarings on Ri to get this power, 
because it can be obtained more simply by means of the formula 


(7.215) Ri = R° — OXUQ,. 


Furthermore, since the actual entries of Rj, or any higher power of Ri, are 
not required for the determination of trial values if the sums of the rows 
are known, additional labor can be saved by means of formula (7.21). The 
values Sí? and Ej; may be considered as elements of the matrices R? and 


Оһ, respectively. Then, according to (7.21), 


59 = SY — NEn, 


so that 52, the sum of the elements in row j of the matrix R?, can be ob- 
tained without calculating the individual entries in Rj. 

Construct Table D.10, in which each block contains the derivation of the 
trial values from the power of Ri represented by the superscript on ajz. In 
the first block the values of S; and Ёл are copied from Tables D.1 and D.7, 
and the sums Sj are obtained by subtraction. Then the trial values а) are 
caleulated by dividing the corresponding sums Sj by the largest one of 
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them (in absolute value), that is, by Sj: = .245. Record the values of Sí? 
from Table D.2 in the second block of Table D.10, retaining only two deci- 
mal places since all the work is based upon three significant figures, and 


TABLE D.10 


DETERMINATION OF TRIAL VALUES FOR THE 
CALCULATION OF THE F, COEFFICIENTS 


Vari- 

able 5i Ej Sá ар sp En 500 аф 

J 
Ti. асс 4.918 | 5.067 | —.149 | — .608 || 22.37 | 22.58| —.21| — .57 
Оо каван 4.844 | 5.014 | —.170 | — .694 || 22.07 | 22.34 | – 27 | — .73 
p 4.601 4.783 | —.182 | — .743 21.04 21.31 —.27 | — .78 | 
p" 4.726 4.873 | —.147 | — .600 21.50 21.71 —.21 | — .57 
бозы» aa 4.656 4.411 .245 1.000 20.02 19.65 .37 1.00 
[m 3.981 3.763 .218 .890 17.10 16.77 .33 .89 
Mises d 3.515 3.312 .203 .829 15.07 14.76 .31 .84 
Bs zs ва 3.831 3.655 .176 .718 16.54 16.29 .25 .68 t 

TABLE D.11 
CALCULATION OF THE F; COEFFICIENTS 

Vari- — 

able в, а вй вн aja аја “а |On jV A/V Iah 

J 
Жете = .57|— .865|— .580- .8856— .5851|— .8852 — .5851 —.828 
ИИ — .18|-1.1001- .737|—1.1152;— .7368 —1.1148,— .7369 —.414 
Boer — .73,—1.097|— .735|—1.1118|— .7345|—1.1112|— .7345 —.412 И, 
4......|— .57|— .902/— .605- ‚9129 — .6031|— .9129|— .6034 — .339 
Bosra 1.00 1.492) 1.000 1.5136 1.0000) 1.5129| 1.0000 .561 | 
Жн .89| 1.348 .903 1.3666 .9029| 1.3660} .9029 -507 | 
шө .84 1.3001 .871| 1.3144| .8684| 1.3142] 8687 .488 
[ .68|  .987| .662 1.0005| .6610| 1.0001|  .6610 .371 


в 
№=1.5129, У/аһ=4.8027 


ігі 


V/V Жа} = -56126 


one additional figure is sufficient to assure the accuracy of the three figures. 
Compute the products Ej, with the value № = 4.4556 taken from Table 
D.6, and again keep four figures. The sums 82 are obtained simply by sub- { 
traction, and the corresponding «f? are then determined. These values are 
truly significant to only one decimal place, and in the one Significant figure 
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they agree with а), If the calculations in the next block were attempted, it 
would be evident that corresponding values of Sí? and АЕ; are equal to 
three significant figures. For example, S(? = 448 and ХЕ, = 448. Hence 
the sums 5/9 (that is, Sf? — ME;,) are insignificant and af? cannot be ob- 
tained. It therefore follows that the best set of trial values for the calcula- 
tion of aj» is a(2. 

17. Employ the numbers af? as the first set of trial values aj by which 
to multiply the matrix В, to obtain af, in Table D.11. The first of these 
values, for example, is given by 


als = .118(—.57) + .118(—.73) + .110(—.73) + . . . —.149(.68) = —.865. 


Divide the values а), by the largest of them (а) = 1.492) to obtain the 
next set of trial values, which are also designated by аҙ for simplicity. 
Multiply В by these new values and divide the resulting a/, by the largest 
one of them to obtain the next set of trial values. Continue this process 
until corresponding trial values in successive sets agree to three significant 
figures. In the computations, after the first or second iterations, keep four 
figures if three significant figures are desired for the factor coefficients. 
"Three iterations of the form (7.12) were sufficient, in the present example, 
for stability in the trial values ajo. 

18. The value aj, = 1.5129 corresponding to asz = 1.0000 is the char- 
acteristic root №. Then the coefficients of the second factor can be computed 
by means of a formula like (7.7), as follows: 


ajoV No 
ао = —————— = 1,2 ,n 
В Vah eh... + oh ч 4 | 


The coefficients аҙ are given in Table D.11. 
19. The final calculation of the coefficients ау can be checked by means 
of the formula 


n 
> a?, = №. 


ј=1 


In the example, the sum of the squares of the eight coefficients is 1.511 and 
№ = 1.5129, so that the coefficients check. 

20. То obtain additional factors proceed with the form of calculation as 
indicated in steps 13-19. The notation of Table D.8 will aid in the applica- 
tion of the method, for then the only changes in the preceding steps will be 
the advancement of the indices. 
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For the given data it will be shown now that no further factors are re- 
quired. The products of the second-factor coefficients әр), = ауа» given 
in Table D.12, are subtracted from the corresponding residuals ir; of Table 


TABLE D.12 
PRODUCT MATRIX: 0, = |a; || 

1 2 3 4 5 6 7 8 
1 | ДО, ation а | 
2 ШЕ И арз залол | 
3 135 .171 Eco ME | 
4 2111 .140 .140 115... 
5 —.184 —.232 —.231 —.190 .315 .... 
6 —.166 —.210 -.209 -.172 .284 .257 .... 
7 —.160 -.202 —.201 —.165 .274 .247 .238 .... | 
8 —.122 —.154 -.153 -.196 .208 .188 .181 138 


D.9 to obtain the matrix of second-factor residuals which is presented in 
Table D.13. These residuals are obviously insignificant and so may be con- 
sidered as final. Inasmuch as the problem of factor analysis is to account for 


TABLE D.13 
MATRIX OF SECOND-FACTOR RESIDUALS: В; 

1 2 3 4 5 6 T 8 
TU EE бе ийй»: ойы rede. aimee ejm 
М эЛ - 005) была: teres чуй» Оты puts 
ug 0 Е Г ou m Sees шалын 
t М0 =D = 00% 018 2 ux чыш. 
5|  .016 —.020 .06  .010 —.003 ...... ...... 
6 ..... 
7| - 25.090: aves 
81 = .011 .058 


the total communality variance, a more definite check on the adequacy of à 
solution is afforded by the extent to which the sum of the contributions of 
the factors agrees with the original total communality. In the present ex- 
ample, two common factors account for practically 100 per cent of the com- 
munality. The percentage contributions of the individual factors are pre- 
sented in Table 7.4, where the complete principal-factor pattern is exhibited. 


APPENDIX E 
OUTLINE FOR COMPUTATION OF A CENTROID PATTERN 


By the method developed in 8.2, a centroid pattern can be obtained for 
any correlation matrix. The details of the numerical caleulations are de- 
Scribed in the following steps. А centroid solution for the first thirteen of 
the twenty-four psychological tests of 2.8 is obtained to illustrate the steps 
in the outline. 

1. Determine the best possible estimates of the communalities for the 
given data. The problem of estimation of communalities is treated in 7.2. 
The choice of communalities for the illustrative example is indicated in 8.3. 

2. Put the estimates of the communalities in the principal diagonal of 
the correlation matrix, writing the correlations in the lower-left half of the 
table only. The intercorrelations of the thirteen psychological tests are 
given in Table E.1, in which the communalities are also included. 

3. The procedure for the calculation of the coefficients of the first cen- 
troid factor is presented in this and the next three steps for the case of a 
matrix involving very few negative correlations. If а matrix contains а 
large number of negative correlations, then the procedure of steps 11-27 
Should be followed. The present example is typical in that it involves a 
matrix of generally positive correlations. 

Obtain the sums of the complete rows (or columns) of the correlation 
matrix. Although only half of the intercorrelations are written explicitly, 
the complete sums can be obtained very simply. Thus, for any variable j, 
add the entries in row j to the diagonal and the entries in column j (includ- 
ing the diagonal value). This sum is represented by 


бұл Ma. 


k 


Where the summation extends over the n values of k, and r;; is the estimate 
of the communality h?. Throughout this appendix the symbol = will be 
used to connote summation from 1 ton when а single index is involved; and, 
When more than one index is involved, the index for which the summation 
is made will be specified. The sums S;, for the given data, are presented in 
Table Ел. 
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4. Calculate the total Т of all the entries in the correlation matrix. This 
may be done simply by use of the formula 


Т-26;. 
For the example, 


13 
= WIS, = 4.592 + 2.687 + . . . + 5.103 = 57.314, 


j=l 


апа Vf = 7.5706, 1/VT = .1321. The sum Т may then be checked by 
means of the formula 


T-22* Tj Хт у 


JSk 


where all indices extend over the range 1 to n, and the first term indicates 
summation for both j and k under the restriction j € k. This check is also 
given in Table E.1. 

Б. The coefficients of the first centroid factor now сап be computed by 
means of formula (8.6), namely, 


ал = УЬ P= Vy Byes vig’) 


The factor coefficient for any variable z; thus is obtained by dividing the 
sum of the n correlations of this variable with all others (including the com- 
munality) by the square root of the sum of all the correlations in the com- 
plete matrix. For a large set of variables, it is convenient to calculate the 
reciprocal of VT and to multiply all the values S; by it. The factor coeffi- 
cients for the given data are obtained by multiplying the respective sums 
S; by 1/VT = .1321 and are presented in the last row of Table ЕЛ. 
6. A check on the preceding computation is available. Since 


5; 
ой = VT 
it follows that 
Di = zan = Je Si ОБТ = VT. 


In the example, 
D; = Хар = 7.571, 


which agrees, except for errors due to rounding, with the value of VT. 
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7. Before additional factors can be determined, the first-factor residuals 
are required. A table of products of the first-factor coefficients may be рге- 
pared to facilitate the caleulation of these residuals, just as in the case of 
step 13 of Appendix D. The elements ip;, (-ары) of the product matrix Q, 
are obtained as follows: the values in the first column are the respective 
products of ац by itself and every other ал (j = 2,3,..., n), the diagonal 
value and all the entries below it in the second column are the respective 
products of as; by itself and all remaining coefficients ал (j = 3,4,..., n), 
and so on for the values in and below the diagonal in all the other columns. 
Since the product matrix is symmetric, it is not necessary to write the values 
above the principal diagonal. The matrix О, = lupis = ||алал|| for the 
given data is presented in Table E.2. For example, the elements in the 
first column of the product matrix are 


ри =a} = (607) = 368, 
ipi? = Auda = (.607) (.355) = 215, 
spia = апаз = (.607)(.418) = .254, 
(Ма = апаа = (.607)(.478) = .290, etc. 


8. To check the calculations of the products, step 14 of Appendix D can 
be applied. Employing the same notation as in the principal-factor method, 


the sums of the complete rows (or columns) of the product matrix О: are 
designated by 


^ “ы В 
Ед = Sanaa (FH, 449 , 
k 
or 
Ел = ajDi, 


which serves as a check on the products of the first-factor coefficients. For 
the given data the thirteen values of Ej and the corresponding checking 
values ал are presented in Table E.2. 


9. Subtract the values in the product matrix О: from the corresponding 
entries in the correlation matrix R to obtain the matrix of first-factor residu- 


als Ry = |17: = |5 — ipis]. The first-factor residuals are presented in 
Table E.3. For example, 


iuo = 7р — apu = .558 — 368 = 1190 A 
1 = T29 — ips = .195 — .248 = —.053, 


and all the other residuals, which are given in and below the diagonal in 
Table E.3, can be obtained in a similar manner. 


= 
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10. A check on the subtractions involved in obtaining the first-factor 
residuals is afforded by 


Sarin. = S; — Ел. 
k 


The sum of all the residuals for each of the variables appear in Table E.3 
and may be checked with the differences S; — Ej. These numbers always 
must check exactly, of course, because no statistical discrepancies, such as 
rounding of numbers, are involved. The sum of the residuals for variable 5, 


for example, is 
Xu —.1224-.026 c... —.147 = .000, 


while S, — Eg = 5.519 — 5.519 = .000. 

The preceding check may be obviated, for in the residual-factor space 
the centroid is at the origin, and hence it is expected that each of the residual 
sums will be approximately zero. Then the foregoing check may be em- 
ployed in individual cases where the difference of a particular sum from 
zero is questionable. Thus, if it is doubted that the largest sum (in absolute 
value) differs from zero only because of rounding errors, the check may be 
applied to it. In the example one of the largest sums is 27; = —.004, and 
S; — En = 5.455 — 5.459 = —.004. 

11. To remove the centroid from the origin in the residual-factor space 
and to increase the contribution of the second factor to the residual vari- 
ance, certain variables are reflected in the origin. The variables to be re- 
flected are determined from a table similar to Table E.4, in which the num- 
ber of negative residuals for each variable is recorded and minimized by 
the reflection of variables. In this table put the numbers of the variables 
in the stub and allow two rows at the bottom of the table for the totals in 
the respective columns and the differences between successive totals, which 
will be useful for checking purposes. The first column* of the table should 
be headed “Reflected Variable,” and a minus sign will be placed in this 
column opposite any variable which is to be reflected. The remaining col- 
umns are introduced successively as an increasing number of variables are 
reflected. 

19. Count the number of negative signs for each variable in the residual 
matrix of Table E.3 and record in the second column and in the appropriate 
row of Table E.4. The second column of Table E.4 is headed “Before Re- 
flection,” because the values put in this column refer to the number of nega- 
tive residuals before any variables are reflected. It should be noted that, 

* Неге, and in the remainder of this outline, the column numbers refer to the body 
of the table, excluding the stub. 
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although only half of the symmetric residual matrix is written explicitly, 
the number of negative signs to be considered for each variable is that of 
the total matrix. In other words, when counting the number of negative 
signs read across the row and down the column for a specified variable. For 
example, the number of negative signs, before any reflections, for variable 
1157. 

13. Add all the entries in the second column of Table E.4 and record the 
sum in the “Total” row. In the present example this sum is 96. The count 


TABLE Е.4 


NUMBER OF MINUS SIGNS FOR FIRST-FACTOR RESIDUALS AFTER 
SUCCESSIVE REFLECTIONS OF VARIABLES 


Hà BEFORE Arter REFLECTION OF SUCCEBSIVE VARIABLES 
VARIABLE FLECTED REFLEC- 

VARIABLE TION 10 11 12 13 1 
Тотса асыны ны = 7 6 5 6 7 5 
Ds impresso, deen ails sexe uer анта ырза 7 6 5 4 5 6 
Bis сел aie ipiius] IF noaa жайы 7 6 5 4 5 6 
PR рр ides Halse is ayy sah cit 7 6 5 4 5 6 
D vases oar iot бы эуен ЭБ А 7 6 5 4 3 2 
prse "sueta zinc od fa tinis cs 8 7 6 5 4 Б] 
oie egt: Scope ar canneries’ 8 7 6 5 4 3 
BUS жасу чайсыз, мин 6 5 4 3 2 1 
авада "р 8 7 6 5 4 Б] 
TO cessi iio xni s - 9 3 2 1 0 1 
Ж шуылды vases 4 = 9 10 2 1 0 1 
Т э завых ғына = 8 9 10 2 1 0 
ET E = 5 6 7 8 4 3 
DOUBT scite ou] сым көзу 96 84 68 52 44 40 
ур Че ЖАНН Резина 12 16 16 8 4 


of negative signs may be checked by re-counting the negative residuals in 


Table E.3, which should be equal to one-half of the total recorded in the 
second column of Table E.4. 


14. Pick the variable with the largest number of negative signs to be 
reflected first. If several variables have the same maximum number of 
negative signs, any one of them may be arbitrarily selected for reflection. 
In the example, variables 10 and 11 have а maximum of 9 negative signs 
and 210 is arbitrarily selected for reflection. қ 

15. Opposite variable 10 іп the first column of Table E.4 place а minus 
sign to indicate that this variable is to be reflected. An adjustment in the 
number of negative signs for each variable will be made as if variable 10 
were reflected in Table E.3, (i.e., as if all the signs for variable 10 were 


> SS — ——————————ÁÁ [LLL o —Á 
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changed) and these results will be recorded in the third column of Table E.4. 
This column is headed “‘10,” to indicate that the count of negative signs for 
each variable is that after variable 10 is reflected. 

16. For the variable being reflected the adjusted number of negative 
signs is (n — 1) minus the number of negative signs it had before reflection. 
Upon reflection of a given variable, every residual which was positive be- 
comes negative and every negative residual becomes positive, except that 
the value in the diagonal of the residual matrix remains unchanged. The 
value (n — 1) is the total number of entries in a given row and column of 
(the lower half of) Table E.3, ignoring the diagonal. In the example, 
n — 1 — 12 and the entry for variable 10 in the third column of Table E.4 
is12—9-3. 

17. It is not necessary to change all the signs of the residuals for the 
variable being reflected in order to count the number of negative signs for 
the other variables after the reflection. Instead, consider the sign of each 
entry except the diagonal in the row and column of Table E.3 for the vari- 
able being reflected, aad proceed as follows. 

a) If the entry for a particular variable, which was not previously re- 
flected, is positive, increase by one the number of negative signs for that 
variable recorded in the second column of Table E.4 and record the new 
value for that variable in the third column of Table E.4. For example, the 
entry for variable 11 in column 10 of Table E.3 is positive, and since 21 was 
not previously reflected, the number of negative signs for it is increased one, 
from 9 to 10, in Table E.4 after variable 10 is reflected. 

b) If the entry for a partieular variable, which was not previously re- 
flected, is negative, decrease by one the number of negative signs for that 
variable and record the new value for that variable in the third column of 
Table E.4. For example, the entry for variable 1 in row 10 of Table E.3 is 
negative, and since 2, has not been reflected, the number of negative signs 
for it is decreased one, and the result 6 is recorded for it in the third column 
of Table E.4. 

The general rules for sign changes are formulated conveniently in 
Table E.5. 

18. The count of negative signs after variable 10 has been reflected can 
be checked by the following procedure. Add all the entries in the third col- 
umn of Table E.4, getting 84; and subtract this total from the preceding 
one, obtaining 12 for the difference. This difference should be twice the 
difference between the number of negative signs for variable 10 before and 
after reflection. In the example, 


12 = 2(9- 3), 
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and so the arithmetical work involved in the third column of Table E.4 is 
checked. 

19. Pick the variable having the largest number of negative signs in the 
third column of Table E.4 as the next one to be reflected. Thisis variable 11, 
which has 10 negative signs. Put a minus sign opposite variable 11 in the 
first column and the number “11” at the head of the fourth column of 
Table E.4 to indicate that this variable is being reflected. Then adjust the 
number of negative signs for cach variable as if variable 11 were reflected 
in Table E.3, and record these results in the fourth column of Table E.4. 
Тһе adjustments are made according to the procedure outlined in steps 
16 and 17. 

TABLE E.5 
RULES FOR SIGN-CHANGE ADJUSTMENTS 


Entry in Row (or Entry in Row (or 
Column) of Re- Column) of Re- 
flected Variable flected Variable 
Is Positive Is Negative 
Not previously reflected (or re- 
flected an even number of Increase one Decrease one 
times) 
Previously reflected once (or any Decrease one Increase one 
odd number of times) 
| 


For example, the entry for variable 1 in row 11 of Table E.3 is negative 
(second column of Table E.5), and, since z; has not been reflected (first row 
of Table E.5), the rule for the sign adjustment is given in the upper-right 
cell of Table E.5. The number of negative signs for z, is reduced from 6 to 
5, which is recorded in the fourth column of Table E.4. Another example, 
that of variable 10, may be considered. The entry for variable 10 in row 11 
of Table Е.З is positive, and, since 210 was previously reflected once, the 
rule for the sign adjustment is given in the lower-left cell of Table E.5. 'The 
number of negative signs for variable 10 is then changed from three to two. 

Check the values in the fourth column of Table E.4 by the method indi- 
cated in step 18. 

20. If zero values should appear in any of the correlation or residual 
tables, they may be treated as positive numbers in making sign adjustments 
for the reflection of variables. The diagonal values of the residual tables are 
not considered in the count of negative signs, for, if a variable is reflected, 
its “self-correlation’”’ remains unchanged. 

21. It may happen that a variable which had already been reflected may 
again appear as the variable with a maximum number of negative signs 
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after several other variables are reflected. In this case the variable is re- 
flected again, changing the minus to plus in the first column of Table E.4, 
and the number of minus signs is adjusted for each of the variables. 

22. Тһе reflection of variables is continued until each variable has less 
than n/2 negative residuals. In the example n = 13, so that the reflections 
are carried to the point where there are six or fewer negative signs for each 
variable. It will be noted that upon reflection of variable 1, the entries in 
the last column of Table E.4 are six or less. 

23. Тһе variables having minus signs in the first column of Table E.4 
now may be actually reflected in Table E.3. In order to make the procedure 
perfectly clear, two additional tables, Е.З” and Е.З”, are included. In prac- 
tice, however, these additional tables may be obviated by incorporating 
them in Table E.3. That procedure is indicated by Table E.7, in which the 
third-factor coefficients are caleulated. 

24. First place а minus sign before the column number of each variable 
which is to be reflected, i.e., before variables 1, 10, 11, 12, and 13 in Table 
Е.З”. This is done for the convenience of the subsequent sign changes in 
this table. 

The signs of the residuals ;r;; may be changed according to the following 
formula: 


Tijk = ejexGris) . 


It will be recalled that the epsilons are merely algebraic symbols for the 
plus or minus signs. Hence, if neither 2; nor 2, was reflected, or if both 
variables were reflected, then rij; = i1rj4; but, if only one or the other of z; 
and z, was reflected, then rij; = —17;к. First go through the upper half of 
Table E.3, one row at a time, and insert minus signs according to the above 
rules. А convenient procedure is to look at each entry of the first column, 
note the adjusted sign, and when this sign should be minus, record a minus 
sign in the corresponding cell of the first row of the upper half of the table. 
Check the total number of minus signs for the first variable with the number 
given for that variable in the last column of Table E.4. Then proceed to 
the second column of the lower half of the table, note the sign changes, and 
record the minus signs in the second row of the upper half of the table. The 
count of six minus signs for variable 2, in the second column and second row 
of the upper half of Table E.3', agrees with that given in the last column 
of Table E.4. Continue this process of sign changes for every variable in 
Table E.3’. As an additional check, the total number of minus signs in the 
upper half of Table E.3' must be equal to one-half of the total given in the 
last column of Table E.4. 

25. Now, merely copy the values (without any algebraic signs) from the 
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columns of the lower half of Table E.3 into the corresponding rows of the 
upper half of Table E.3. This is done in Table E.3" for the example. The 
values so obtained are the residuals of the reflected variables. 

26. Тһе second-factor coefficients are calculated from the values in the 
upper half of the table.* Тһе sum of the residuals (after reflection) for 


any variable z; is 
Sa = > Tiik; 
k 


and the total of all the reflected residuals is 
Т = ZSi E 


The thirteen sums Sj; and the total T, are presented in Table Е.З”, The 
square root and the reciprocal of the square root of T, are also given. The 
algebraic signs of the sums Sj; are changed for the variables that have been 
reflected, and the resulting values, 15; = ¢;Sj, are presented in Table E3”, 
Finally, by multiplying each 18; by 1/ VT, the respective coefficients аҙ are 
obtained. These are the coefficients for the observed variables, not the re- 
flected ones. 

27. A useful check on the calculation of the second-factor coefficients 
(and the coefficients of all other factors except the first) is given by the fact 
that the sum of the coefficients must be approximately zero if a centroid 
system has been obtained. In the example, 


D: = Za; = .000. 


Another check, like the one given in step 6 for the first factor, follows from 
the relation 


"C CN. 
єй = VT, УТ, Т 


The sum of the factor coefficients for the reflected variables must be equal 
to the square root of the total of all the reflected residuals, The coefficients 
for the reflected variables are merely those given in the last row of Table 
E.3" with the signs changed for all variables which have been reflected. 
Thus, 


Zejaj; = .060 + .038 + .148+...4 368 = 3.706 , 
which agr 2s with the value of УТ. 


* It may be noted again that this portion of the table was separated out as Table E..3/ 
for clarification, but in practice only one Table E.3 need be made. 
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28. Proceed, asin steps 7 through 27, to determine the third-factor coeffi- 
cients. The calculations leading to a;; are given in Tables E.6, E.7, and E.8. 
This procedure is repeated over and over again until sufficient factors are 
obtained, which account for the total communality. 

It is evident from the third-factor residuals, given in Table E.10, that 
three factors are sufficient to account for the original correlations. Further- 
more, the sum of the contributions of the three factors agrees well with the 
original total communality. This is indicated in Table 8.1, where the cen- 
troid pattern for the thirteen psychological tests is presented. The percent- 
age contribution of each factor to the total communality is also given in 
'Table 8.1. The discrepancy between the original and caleulated commu- 
nality for each test is given in the last column of Table 8.1. The sum of the 
thirteen differences is —.030, or —0.4 per cent of the original total commu- 
nality (6.936). This indicates a very good factorization, with a slight tend- 
ency for overfactoring. 
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ТАВГЕ Е.8 


NUMBER ОҒ MINUS SIGNS FOR SECOND-FACTOR RESIDUALS 
AFTER SUCCESSIVE REFLECTIONS OF VARIABLES 


AFTER REFLECTION оғ Suc- 
Re- ВЕРОВЕ СЕВВІУЕ VARIABLES 
VARIABLE FLECTED Re- 
VARIABLE | FLECTION 

1 2 4 13 3 

8 4 3 2 1 0 

7 8 4 3 2 1 

6 7 8 9 10 2 

7 8 9 3 2 1 

4 3 4 3. 2 1 

7 6 5 4 3 2 

6 5 4 3 2 1 

6 5 4 5 4 5 

7 6 5 4 3 2 

5 4 3 2 1 0 

7 6 5 4 5 4 

7 6 5 4 3 4 

7 8 9 10 2 1 

als ja 76 68 56 40 24 
Difference. . 8 8 12 16 16 
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APPENDIX F 


EVALUATION OF THE ROOTS OF А 
POLYNOMIAL EQUATION 


In obtaining a derived principal-factor pattern, the solution of a poly- 
nomial equation of the third, fourth, or higher degree is required. Several 
methods are available for computing the roots of such an equation to any 
desired number of decimal places. Each method, however, requires some 
preliminary information concerning the root to be calculated. It is neces- 
sary to know the limits between which the root lies and that there is no 
other root between the same limits. Thus it would be sufficient to know 
that the root is between 5 and 6, provided that there is no other root be- 
tween these limits. If there were another root within this range, narrower 
limits would be necessary. When limits are obtained such that only one 
root is contained in the interval, then the root is said to be isolated. 

As the first step toward the isolation of the roots of an equation of the 
mth degree, 


(ЕЛ) Ли) = u” + aw + азы" +... + asap d as = 0, 


Descartes's rule of signs will be presented. The term ‘‘variation of sign" 
is used to indicate that two consecutive terms of a polynomial or equation 
have coefficients of unlike signs; and by the variations of sign of an equation 
is meant all the variations presented by consecutive terms (when written 
in order of decreasing powers of the variable). 

Descartes’s RULE. The number of positive real roots of an equation f(u) = 
0 is either equal to the number of its variations of sign or is less than that number 
by a positive even integer. The number of negative real roots of f(u) = 0 is 
either equal to the number of variations of sign of f(—u) = 0 or is less than 
that number by a positive even integer. A root of multiplicity q is here counted 
as q roots. 

Direct application of Descartes's rule of signs will not isolate the roots of 
an equation, except to give an upper limit to the number of positive and 
negative roots. Descartes’s rule may be used as a rough check on the deriva- 
tion of the general characteristic equation (9.10), since all the roots of this 
equation must be real and positive in factor analysis. This check consists 
simply in noting that, when the characteristic equation is written in order of 
decreasing powers of the variable, the signs of the terms must be alternately 
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plus and minus. The number of variations of signs is then equal to the de- 
grec of the equation, giving the number of positive real roots. Thus, if the 
general equation (Е.1) is a characteristic equation, then the coefficients 
must be alternately positive and negative, and the equation may be written 
in the form (9.10). 

For purposes of illustrating the methods of evaluating the roots of a poly- 
nomial equation, the third example of 9.3 will be considered. The charac- 


teristic equation arising in the process of transforming the averoid to the 
principal-factor solution is 


(F.2) Ди) = wt — 11.014243 + 30.5260,? — 31.7349, + 11.1727 = 0. 


In this equation there are four variations of sign, indic 
real roots. 

More exact methods for isolating the roots of a polynomial equation 
f(u) = 0 will now be presented. One procedure is to plot the polynomial 
f(u), and by means of this graph to isolate the roots. To obtain a reliable 
graph, however, the bend points of the curve are required, and, since these 
points are difficult to obtain for a polynomial of degree greater than three, 
this method is usually impracticable. A much more effective, analytical 
procedure is given by Sturm’s method.* 

The first step in Sturm’s method is to divide Ха) by the first derivative 
f'(u), obtaining a remainder r(u), whose degree is less than that of f’. De- 
noting the quotient by qı, this result may be written / = фу +r. Nowin- 
dicate the negative of this remainder by fs(u), so that f = mf’ — fs. Then 
divide / by f; and denote the remainder with its sign changed by /з(и). 
Continue this process until a constant remainder is obtained. This pro- 
cedure may be exhibited as follows: 


ating four positive 


f=af' -he, 
ГЛ-аел-Л, 
(Е.3) Љ = аја = fa, 


t3 
Jn- = Qn-ifn-i ==аз 


where fm is a constant. 

If fm = 0 then f,, , divides fm—2 and all the other polynomials f;, includ- 
ing f’ and f, as may be seen by employing equations (F.3) in reverse order. 
Conversely, any common divisor of f and f' divides f; and hence all the other 
polynomials. Thus if fm = 0, then fn_i is the greatest common divisor of 

* L. E. Dickson, First Course in the Theory of Equations, pp. 75-78. 
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f and f’, and a root of f, a(u) = 0 is a multiple root of f(u) = 0.* On the 
other hand, if fm is a constant different from zero then f and f' have no com- 
mon divisor involving и, and f(z) = 0 has no multiple root. In the remain- 
der of this appendix it will be assumed that f(z) = 0 has no multiple roots, 
so that in the sequence of polynomials, f, f^, fo, . . . , fm, the last one, fm, is 
а constant different from zero. 

Srurm’s THEOREM. If a and b are real numbers, а < b, neither a root of 
J(u) = 0, the number of real roots of f(u) = 0 between а and b is equal to 
the excess of the number of variations of sign of 


(F.4) Se), f(a), f), «a foi, Jm 


for и = a over the number of variations of sign for u = b. Terms which van- 
ish are to be dropped out before counting the variations of sign. 

The application of Sturm's Theorem to the isolation of the roots of 
equation (F.2) will now be given. The first derivative of the function in 
(F.2) is 


/'(и) = 4u? — 33.0426, + 61.05204 — 31.7349 , 


and the first of equations (F.3) becomes 


J = (.2500un — .6884)f’ — fo, 
where d 


fo = 7.48354? — 18.22704 + 10.6736 . 


By dividing f’ by fz, the second of equations (F.3) is obtained, as follows: 
/' = (.5845и — 3.1136)f2 — fs, where f; = 1.40464 — 1.4984. 


Continuing this process, it is found that on the next division the remainder 
is à constant different from zero. The sequence of polynomials (F.4) may 
then be summarized as follows: 


f = wt — 11.014243 + 30.52604? — 31.73494 + 11.1727 , 
f! = 4u? — 33.04264? + 61.05204 — 31.7349, 
(F.5) | fy = 7.48354? — 18.2270, + 10.6736, 
1.4046 — 1.4984, 
fa = 2615. 


- 
D 


* Ibid., p. 61. 
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The roots of (и) = 0 may then be isolated conveniently as indicated in 
Table F.1. 

In selecting trial values of the variable и, integral values are usually 
taken, beginning with zero, one, and зо on. According to Sturm's Theorem, 
there is опе real root of f(u) = 0 between 0 and 1 since 


Vo-Vi24—-321, 


where V, denotes the number of-variations of sign of the numbers (F.4) 
when и is а particular real number not a root of f(u) = 0. Trying the next 
integer—2—it is found that the excess of the number of variations of sign 
of the polynomials (F.4) for u = 1 over the number of variations of sign 
for р = 2 is two. Consequently, there are two roots between 1 and 2, and 


TABLE F.1 
ISOLATION OF ROOTS or f(u) =0 
Brana NuwnEn 
TRIAL оғ 
VALUE | VARIA- COMMENTS 
TIONS 
А / | r | љ л | Se | Va 
EA ЖЫ esi] + | -|+ 4 One root between и=0 and и=1 
liene Say) 6 = 5 3 
De шал ste —-|-—-|ct|i-c-|-c 1 Two roots between 4 —1 and ш-2 
OR AREO +|-|+ ++ 2 | One root between u —1.5 and 42, and 
| | one root between и=1 and u=1.5 
Жэ. чөй Б -|+| + | +/+!) 1 | No roots between и=2 and һ-7 
Bs usus +| + | + | + | + 0 One root between и=7 and и=8 


hence these roots are not sufficiently isolated. A value of и halfway be- 
tween these numbers is arbitrarily taken as the next trial value. It is then 
found that Vi — Vi, = 1 and Vis — V2 = 1 so that one root is isolated 
between 1 and 1.5, and the other between 1.5 and 2. 

At this point, three roots are already isolated. Their total contribution 
to the communality (equal to the sum of the three roots) may be estimated 
to be less than 4. From the averoid solution of Table 8.10 it is known that 
the total contribution of the four principal factors will be about 11. Тһеге- 
fore, the remaining factor must contribute upward of 7 to the total com- 
munality, or the remaining root of (F.2) is probably greater than 7. Hence, 
after isolating the three roots in Table F.1, the next trial value of wis taken 
ав 7. The last root is isolated between и = 7 and р = 8. 

It may be noted that 


(F.6) Ve Z Ve for a«b. 
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This relation may serve as a rough check on the computation of Sturm’s 
functions (F.4). 

When a particular root of an equation is isolated, it can be computed to 
any number of decimal places by means of Horner's Method.* This method 
involves successive transformations or reductions of the original equation. 
Thus, to find the root between 7 and 8 of equation (F.2), set u = 7 + d, 
where d is а decimal fraction to be determined. Direct substitution into 
(F.2) gives the transformed equation for d: 


(F.7) а* + 16.9858d3 + 93.2278d? + 148.5417d — 92.0682 = 0. 


То obtain an approximation to the decimal d, ignore for the moment the 
terms involving d to the second or higher degree. Then from 148.5417d — 
92.0682 — 0, d — 0.6. But this value is too large, since all the terms ig- 
nored are positive. For d — 0.5, the polynomial in (F.7) is still positive, 
while for d = 0.4 it is negative. Hence d = 0.4 + h, where h is of the de- 
nomination hundredths. The value of the root of (F.2) may then be writ- 
ten as и = 7.4 + h, or, to simplify the notation, и = 7.4 + d, where d is 
now a new decimal of the denomination hundredths. 

Тһе method as just described is laborious especially for equations of high 
degree. If synthetic division} is employed instead of direct substitution, 
however, the work can be organized in the simple form indicated in Table 
F.2. In this table, the coefficients and constant term of equation (F.2) are 
recorded in the first row, and the equation is designated by (1). The given 
polynomial is divided synthetically by » — 7, and each quotient is also 
divided by и — 7. Then the successive remainders are the coefficients of 
the transformed equation (F.7). These coefficients appear in bold-face type 
just under the first zigzag line in Table F.2, and the fourth-degree equation 
with these coefficients is labeled (2). The polynomial (2), again expressed 
in terms of the variable и, is divided synthetically by и — .4, and each quo- 
tient is also divided by и — .4. The remainders are the coefficients of the 
next transformed equation, which is designated by (3) in the table. Again 
consider the variable in the transformed equation as d and ignore the terms 
in d?, d*, and d*in order to get an approximation to the next decimal. Then 
from 231.5331d — 16.6224 = 0, d = .07. The polynomial (3) is positive for 
d = .07 and negative for d = .06. Hence it is divided by .06. This process 
is continued, as completely illustrated in Table F.2, until the root of the 
original equation is caleulated to as many decimal places as desired. In 
the present case the root is given correctly to four decimal places by ш = 
7.4694. 


* Ibid., pp. 86-89. 1 Ibid., pp. 13-15. 
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To clarify further the procedure for evaluating a root of an equation, the 
complete work for the second largest root of (F.2) is presented in Table F.3. 


TABLE F.2 
CALCULATION OF ш BY HORNER'S METHOD 
иќ и? и? н Constant 
(1) 1 — 11.0142 + 30.5260 — 31.7349 + 11.1727 |7. __ 
i — 28.0094 + 16.9862 — 103.2409 m=7+d 
1- 40142 + 24266 — 14.7487) — 92.0682 
7. + 20.9006 + 163.2904 | 
I+ 2.9858 + 23.3272| + 148.5417 
%; + 69.9006 . 
PER REN = ford = 4 
1+ 90858 + 93.2278 020682 _ (2) ік — for іе 
7 1485417 ^ 6 (2) is + ford =. 
3 i (2) is + for d = .6 
(2) 1 + 16.9858 
4 + 6.0543 + 40.0728 + 75.4458 [0.4 
1 + 17.8808 + 100.1821 + 188.6145| — 16.6224 m=74+d 
4 — - 7.1143 + 42.9186 
1 + 17.7858 + 107.2964] + 231.6331 
4 + 72743) 
1 + 15.1858 + 114.5707 16.6224 _ 07 (8) is — ford = .06 
EMEN WE 231.5831 ” (3) is + for d = .07 
(3) 1 + 18.5858 
06 + 1.1187 + 6.0414 + 14.3085 10.06 
1 + 18.6458 + 115.6894 + 238.4745) — 2,3139 м = 7.46 + d 
206 + 1.1223 + 7.0087 
1 + 18.7058 + 116.8117) + 245.4832 
06 + 1.1259 
1 + 18.7658] + 117.9376 23139 — 009 (4) is — for d — .009 
Ийе Си 2454832 ` (4) is + for d = .01 
(4) 1 + 18.8258 
009 + — .1695 + 1.0630 + 2.2189 |0.009 
1 + 18.8348 + 118.1071 + 246.5462) — .0950 m = 7.469 +d 
—_ 009 + 1696 + 10645 
1 + 18.8438 + 118.2767) + 247.6107 
009 + 1697 
1 + 18.8528) + 118.4464 0950 _ 00038 (5) is — for d = .00038 
.009 247.6107 i (5) is + for 4 = .00039 
(5) 1 + 18.8618 


0004 + 0072 + 050- 001 [0.00038 
1 + 18.8622 + 118.4536 + 247.0587 — -0009 


ш = 7.4694 


One important distinction may be noted between the work in Table F.2 
and that in Table F.3. Whereas the constant terms in the successive trans- 
formed equations of Table Е.2 are all negative, those in Table F.3 are all 
positive. In obtaining an approximation to the decimal at each stage, a 
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different rule must then be applied. In the case of a negative constant 
(Table F.2), the value d is selected which makes the polynomial negative 
just before turning positive. When the constant term is positive (Table 
F.3), however, the value d is selected which just leaves the polynomial 
positive before turning negative. 


TABLE F.3 
CALCULATION OF 4; BY HORNER'S METHOD 
ГЫ Pu и? в Constant 
(1) 1 — 11.0142 + 30.5260 — 31.7349 + 11.1727 |1.5 
1.5 — 14.2713 + 24.3821 — 11.0292 ш = 1.5 +а 
1— 9.5142 + 16.2547 — 7 + 14356 
1.5 — 12.0213 -- 6.350] 
1— 8.0142 + 4.2334) — 1.0027 
15__ 9.7713 
1- 6.5142| — 5.5379 1435 _ 1 (2) is + for d = .09 
1.5 10027 ^ (2) is — ford = .1 
(2) 1— 6.0142 
.09 - 244832 —  .5383 — .1387 [0.09 Ҙ 
1- 4.9242 — 5.9811 — 1.5410] + .0048 из = 1.59 + а 
09 - 4351 — .5775 
1- 4.8342 - 6.4162| - 2.1185 
.09 —  .4270 
1— 4.7442] — 6.8432 0048 _ (02 (3) іѕ + for d = .002 
.09 2.1185 А (3) is - for d = .003 
(3) 1— 4.6542 
002 — .0093 — .0137 — 0043 |0.002 
1- 4.6522 — 6.8525 - 2.1322| + .0005 ш = 1.592 +d 
002 — .0093 - (0197 
1- 4.6502 — 6.8618] — 2.1459 
002 - .0093 
1- 5 - 6.8711 0005 — 00093 (4) is + for d = .00023 
a 12 2.1459 А 4) is — for d = .0009 
(4) i— 4.6463 шайы 
0002 —  .0011 — 0016 - 0005 |0.00023 
1- 4.6460 - 6.8722 - 21475 + .0000 ро = 1.5922 


The two remaining roots, as isolated in Table F.1, сап be computed in 


& manner similar to that of Tables F.2 or F.3. The values of the four roots 
may be summarized as follows: 


щ = 7.4694 , 
(F.8) u = 1.5922, 
из = 1.0934, 
шщ = 0.8591. 
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Тһе sum of these roots is 11.0141, which agrees with the total contribution 
(11.015) of the four averoid factors of Table 8.10. This is to be expected, 
because the total contribution of the four principal factors must be equal 
to the total contribution of the averoid factors, the former solution being 
merely a rotation of the latter in the same common-factor space. Тһе rela- 
tive size of the contributions of successive factors is different, however, in 
the two cases. 


APPENDIX С 
SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS 
G.1. General Procedure 


The problem of solving a set of simultaneous linear equations arises in 
at least four distinct phases of factor analysis, which will be discussed in 
the following sections of this appendix. In general, a system of n equations 
of the first degree in n unknowns can be solved by means of determinants.* 
While the determinantal method has some undisputed theoretical advan- 
tages, a more economical procedure is desired, especially when dealing with 
a large number of variables. The systems of equations which appear in 
factor analysis have symmetric determinants of coefficients and so lend 
themselves to special methods of solution. Gauss’s method of substitution] 
produces à routine scheme for the solution of such a set of equations, in- 
cluding a complete check on the arithmetical work. 

For convenience of the theoretical work, a system of only three equations 
in three unknowns will be employed. The procedure illustrated, however, 
is applicable to a system of any number of equations, provided the determi- 
nant of coefficients is symmetric. Let the equations be 


aX+bY¥ +‹7—-=0, 
(G.1) bX +d¥ +eZ—h=0, 
cX +e¥ +fZ—-—k=0, 


in which X, Y, Z are the unknowns and the remaining letters are constants. 
Solving the first of these equations for X, 


(G.2) Faw Fate, 
a a a 


and substituting this value into the last two of equations (G.1), they become 
б-®т+(-®Ю4-(6-®). 
(G.3) a a 
(8) 022-62) 
a a a 


* See, e.g., Maxime Bocher, Introduction to Higher Algebra, 1935, p. 43. 

+ This method has been referred to as the “Doolittle Solution" in many textbooks on 
statistics. Convenient forms for the solution of a set of normal equations, arising in the 
problem of curve-fitting, were devised by M. H. Doolittle and are presented in T. W. 
Wright and J. Е. Hayford, The Adjustment of Observations, 1906, pp. 101-24. 
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Let 


2 bc c bg a pe, MOO 
4-à-P, а=е- 2, Л-1-6, uoa, қ-ь-%, 


then equations (G.3) may be written in the simpler form 


ФУ + е2 = №, 


с {кт hi wh. 


From the first of these equations, 
e hı 
(G.5) Ү--42%7. 


and, upon substituting this value іп the second equation, it becomes 


е? e 
Setting 
1 eh 
h=h-Ẹ and в=й—7, 


the value of Z may be written explicitly as follows: 


(G.7) 2h 


Then substituting this value of Z into (G.5), the value of Y is determined, 
and, finally, using these values of Y and Z in (G.2), the unknown X is 
determined. 

The substitutions of the last paragraph can be put in a routine outline 
form as indicated in Table G.1. In this form the coefficients and constant 
terms of equations (G.1) are recorded in the appropriate columns of lines 
1, 3, and 7. Since the coefficients of X, Y, and Z in (G.1) constitute a sym- 
metric determinant, and since the method of solution causes terms to the 
left of the principal diagonal to become zero as the variables are eliminated, 
these terms are omitted in Table G.1. If there were more equations in the 
system, there would be additional blocks in the table, each successive block 
including one additional line of calculations. 

The directions for the remaining lines of Table G.1 are self-evident. 
Specific directions for checking the arithmetic, however, have not been in- 
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cluded. The sum of the entries in each line should be obtained. This value 
should agree, except for rounding errors, with the corresponding expression 


TABLE G.1 
OUTLINE FOR GAUSS'S METHOD OF SUBSTITUTION 
Line L, E z | 2 Constant Check Directions 
| 
Forward Solution 
| 
ымы ығай а b с | —g Zi From (G.1) 
Ө Ea асан сер =1 | 25 =f 4 £ -lz labi 
a a | а а а 
a алы ы ЭЛИ а е —h I: From (G.1) 
b? 
ен Маған SHOES Ы Ж йе шы 
а а а а а 
Вјен — di а | —h zi L; + Li 
Saan p Быны ap № ep | athe 
1 di | di 72% zr Ls 
Жыга ар» a Lh otov see OP y Lb Zs From (G.1) 
— RET ава 24 2 |-iw| -fLim'n-a-b 
2 
Bois cep е кан es sacs = ЕСА РЕ ала 
. T. di q di Ls i E? = 5-0 
||: ise dcus cia aes oo ағанды Л =k: zi Li Н Ls + Ls 
ky 1 
асс шына ИЙРЕ. ЧҮҮ -1 = -т> -= Lr 
| fa л? А" 
Back Solution 
From Li...... Z 2h Entry in “Constant” 
А column 
From L,...... ЖЕТЕ h Use entries in “Z” and 
3 а” PA “Constant” columns 
z b. с Use entries in “Y,” “Z,” 
Prom Тас); ку Х-- a! - Ge +2 and “Constant” col- 
umns 


in the check column. The entries in the check column for lines 1, 3, and 7 
are merely the sums of the elements in these lines and are dhecked if the 
remaining checks (employing these values) are satisfied. For example, the 
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sum of the elements in line 5 is (di + е — hı), and this should agree with 
the check sum 


Now, if this check is satisfied, and the sum of the elements in line 4 is 
checked, then the sum Z; must be correct. 

In the back solution of Table G.1 the actual values of X, Y, and Z are 
determined. The value of Z, as given by equation (G.7), is obtained di- 
rectly from the last line of the forward solution. Knowing the value of Z, 
the determination of Y can be effected by means of the values given in line 6 
and agrees with formula (G.5). Finally, employing the values of Y and Z, 
the value of X as given in equation (G.2) сап be obtained from the entries 
in line 2 of Table G.1. After Y, Y, Z have been caleulated, the back solu- 
tion can be checked by substituting these values into (G.1). 


G.2. Application to Solution of Derived Principal-Factor Paltern 


In transforming the averoid solution of Table 8.10 into a principal-factor 
pattern, systems of equations of the form (9.8) must be solved for the direc- 
tion cosines of the principal factors. To illustrate Gauss's method of sub- 
stitution in this situation, consider the following system of equations for 
the direction cosines of the first principal factor: 


—.0288An — .313614 — .2753^4 + .036044 = 0 
(G.8) —.3136X, — 6.0744, + .0413A5 — .3222 = 0 


—.2753^n + 041324 = 6.3707Х% тз 0917Х4 - 0 
-0360A — .3222&4 — .0917A4 — 6.389544 = 0 


, 
D 
, 


, 


where the coefficients are taken from (9.22) with the value of u (= 7.4694) 
given by the largest root in (F.8). 

The rank of the matrix of coefficients of (G.8) is three, and the first three 
equations may be used to solve for the first three unknowns, ^u, №, Аз, 
in terms of the last unknown, Aa. For all values of the latter, the expres- 
sions for the three unknowns will satisfy the given four equations. Then, 
by employing the auxiliary condition, 


(G.9) Mit Mit Mi + I, 


unique values for the four unknowns can be determined. 
The forward solution of the first three of equations (G.8) by Gauss's 
method of substitution is presented in Table G.2. In lines 1, 3, and 7 of 
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this table are recorded the coefficients of the first three of equations (G.8). 
The constant terms are .0360Au, —.3222A4, and —.0917Aq, but for sim- 
plicity Ха is omitted in each case and put at the head of the column. The 
directions for calculating the elements in the other lines of this table are 
similar to those given in Table G.1. 


TABLE G.2 


FORWARD SOLUTION FOR DIRECTION COSINES OF 
First PRINCIPAL-FACTOR AXIS 


| 
Lino ka si ^n oce Check 
— .0288 | — .3136, - .2753 .0360 -  .5817 
=i; | —10.8889 | —9.5500 | 1.2500 —20.1979 
= 6.0744 | ‚0413 — .8222 | — 6.3553 
3.4148 | 2.9977 | — .3920 6.0205 
- 2.6596 , 3.08900 | - .7142 — .8348 
= d. | 1.1427 — .2085 - .1258 
ЭИА aot init er Bene | —6.3707 = .0917 — 6.4624 
Bru aacra SR qu em ad енін | 2.6316 — 23441 2.2875 
Ds copo aer eee me m nta Dai AER ac | 3.4727 | — .8161 2.6506 
Wn dne еее oc P | — .2664 —1.2519 — 1.5183 
Morro аланың | ort ipao аи | Oana СЕН | =e — 4.6993 — 5.6993 


The values of An, №, and № in terms of Ха сап be obtained by means 
of the back solution. From line 11 of Table G.2, Xa = — 4.69934. Then, 
employing this value, the calculation from line 6 becomes 


№ = 1.1427As1 — .2685\ = —5.6384Ми. 


From line 2, 


Au = — 10.88892 — 9.559031 + 1.250054, 


and using the values of ^s; and Хз just calculated, this expression reduces 
to An = 107.57Ма. Substituting these values of Xi, An, and № into equa- 
tion (G.9), this condition becomes 


116964, = 1, 
from which, 


Ха = +.0092744. 
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Using the positive value of Ха, all the direction cosines are determined 
uniquely, as follows: 


Au = .9976, 
An = —.0523, 

10 
(G10) Aa = —.0436, 
Au = 20093. 


There are several checks which may be applied. First of all, the sum of 
the squares of the direction cosines (G.10) must be equal to unity, i.e., 


4 
Y = 9999. 
8-1 


Then, of course, the \’s must satisfy the equations from which they were 
solved and also the fourth equation of (G.8). Employing the values (G.10) 
in equations (G.8), the left-hand members become .0000, .0000, .0001, and 
— 0027, respectively. These values are considered sufficiently close to zero 
to afford a satisfactory check. The greater discrepancy in the check of the 
fourth equation is due to the fact that the latter was not employed in the 
solution of the A's. Hence, the values (G.10) may be accepted as the direc- 
tion cosines of the first principal axis with respect to the four averoid axes. 

The direction cosines of the remaining principal axes can be obtained in 
a similar manner by employing the values of из, из, and py of (F.8) succes- 
sively in (9.22). Three sets of equations of the type (G.8) are thus obtained, 
and from these and auxiliary conditions like (G.9) the values of Asay Аз, 
and às, (s = 1, 2, 3, 4) сап be determined. The complete sets of direction 
cosines are given in Table G.3. Upon multiplying the averoid pattern of 
Table 8.10 by this matrix of transformation, the derived principal-factor 
pattern of Table 9.1 is obtained. 


TABLE G.3 
MATRIX OF TRANSFORMATION 


Py P: | Pi Ps 
.9977 | —.0543 | .0293 0 

.0523 .8274 2603 ; 152 
— .0436 —.1395 | .9533 2660 
0098 5413 | —.1500 ‚8270 


6.3. Application to the Derivation of a Pattern from a Structure 


After a structure, in terms of correlated factors, is obtained, there still 
remains the problem of determining the factor pattern. As indicated in 
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step 6 of 11.3, the coefficients of the factor pattern may be calculated from 
sets of equations of the form 


bi F bn Pio ne bf. m s 
Батууу, + bis БРТ. binry irm = т, 


(G.11) 

bar, s, F bises, F- -F bim zit 
Foreachj(j = 1,2, ... , n) there are m such equations for the determina- 
tion of the coefficients bi, bj2, . . . , bjm of the m common factors. The terms 


in the right-hand members of these equations are the known structure ele- 
ments. 

The calculation of the b’s for any variable 2; can be performed by Gauss's 
method of substitution as indicated in Table G.1. Furthermore, since the 
determinant of coefficients of (G.11) is the same for all values of j, the work 
can be so organized that the solution for all the b's can be made simultane- 
ously. For the illustrative example of 11.3 there are just two factors and 
eight variables, plus two composite variables. The factor coefficients for 
these variables are calculated in Table G.4. The determinant of coefficients 
of the unknowns (with the elements below the diagonal omitted) appears in 
lines 1 and 3 of Table G.4, while the constant terms of equations (G.11) are 
also recorded in lines 1 and 3 for each variable z;. In the directions these 
elements are designated as correlations with y; or ys. The directions for 
the calculation of the other elements in the forward solution are similar to 
those given in Table G.1. 

Тһе back solutions for the values of b п and bj; are calculated from lines 6 
and 2 in the manner indicated in Table G.1, with one exception. Since the 
constant terms are not brought over to the same side of the equation, as is 
done in (G.1), the negatives of the values in the constant. columns of lines 


6 and 2 must be employed. Thus, for j — 1 the factor coefficients are 
given by 


be = .051 
bu = -.4845; + .919 = .894. 


In a similar manner, all the other factor coefficients are calculated. The 
values б, bj; may be checked upon substitution into equations (G.11). For 
the illustration, the coefficients of the first variable may be checked by sub- 
stituting b; = .894 and 6, = .051 in the expression 


bu + бз, 


| 
sais |426, |080: |090- lezo- 00: 68 ceo: 996' +68 фо Jew б eur шол 
| 
iaai Or6' |000: 589: (69/7 іш 086: |600 jeso"— c0 — |160: “д 5522277 g өші шог 


uoijn[og xovg. 


| | | 

001:4- 0%:- 000: (689°— |69/:- 98 — loes- ezo: — zs0° М0 (о-| I= |77” (992 =T |" 9 
ree |002: looo с 686: 20: сш |40: 00°— 120:- 660: 9 |7 прера E 
ogre- |62- lez} — leser- hier- histr- |42- jaer — jeer: — jost- |ep jiez — | 7| TBR р 
бє, 0б lese | (О 6187 zeot Se 6687 (СЕР а p Deed Wi [s puis в 
I£U'$— leg- /6°— jerk- feie- ме- сер gos — 106°— Е6- |616°— (|ы — | '1— We 2 
rere кт о ив ик [ик (ЫЕ с feos’ 1060 606 616 ger. 1 fug, Suse] 

| | (жаға 
uonnjog panso 
oan tn іп | az 12 “ “ ” | “ tz 12 
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вікуінко;) 
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and noting that the result 


.894 + .051(.484) = .919 


agrees with my, . 


G.4. Application to the Estimation of Factors 


Gauss's method of substitution сап be adapted to the solution of the re- 
gression coefficients in the estimation of factors. The linear regression of any 
factor F, on the n variables is given by 


(12.6ы») F, = Baz + Booze +... + Bass (в = 1,2,..., т). 
The normal equations for the determination of the 975 are 


Ba + 7128.2 +... + таба = s, 
(12.755) Taba + Bart... + Tanba = los , 


тыа + 728.2 +... + Ban = tne y 
where tj, = r.z, The coefficients of the 8's are the elements of the sym- 


metric matrix of observed correlations. Then the scheme for Gauss's meth- 
od of substitution of Table G.1 can be followed in the present case. 


Since each set of normal equations for the estimation of the successive 
common factors involves the same matrix of coefficients, all the factors can 
be estimated simultaneously. Furthermore, this means that several sets of 
factors, obtained by different methods of analysis, can be predicted at the 
same time. To illustrate, the two principal factors and the two oblique fac- 
tors for the example of eight physical variables are estimated. The com- 
plete work, including checks, is shown in Table G.5. The observed correla- 
tions are taken from Table 7.1; the correlations of the variables with the 
two principal factors P; (General Physical Growth) and Р, (Body Type) 
are taken from Table 7.4; and the correlations with the two oblique factors 
yı (Lankiness) and ү» (Stockiness) are taken from the factor structure of 
Table 11.2. 

After the 8'з are determined in the “back solution,” the equation of esti- 


mation of any factor may be written. Thus, for the second principal factor, 
the equation of estimation is 


P, = — 261z, — .2672, — .2472; — 11624 + .6062; 
+ .2072, + .0882; + .1612, 


TABLE G.5 
ESTIMATION OF FACTORS 


VARIABLES 


PRINCIPAL 
Factors 


OBLIQUE 
Factors 


5 | 6 | T 


ШЕ 


"^ 7% 


CHECK 


Forward Solution 


ШӘсхоа|ооев|ы- 


ПЕТИ 


- 


(X EN E 


CETL EV 
РА 
wo 
55 
ча 


D 
sa 
ә 
Ф 


Back Solution 


— ее (ев = element in col. з).. 
— .132 — 


— еа 
748Bes— . 75985 —. 5058n ee " 


1608--.0468е--...- 04685 — es 
From line 11 1908— .0768:54- ... + 1338« — es 
From line 6: 8e — 704853 — 340843. . :3248u —es 
From line 2: Ba = — .8168«— .8058:3d- ... — .3828n— es 


>. 
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and a particular value of this factor can be obtained by substituting the 
values of the variables for a given individual. The 87, being a solution of a 
system of normal equations of the type (12.7), must satisfy such equations. 


Hence the “back solution” ean be checked by substituting the 8's in equa- 
tions of the type 


(G.12) Вата + Baria +... + Bastis = Т.Р, (2 £12, Diy 


where F, is any one of the factors. Employing the first line of Table G.5 
(that is, j — 1), this eheck for each of the four factors is given by 


в 
> Beans = 858 -тыр,, 
kel 


8 
> Вь,атік 
kal 


—.328 = Та\Р 


2? 


8 

Senne = 919 = ray, , 
k=1 

8 
D Bhat = 484 «ет... 
-1 


Аз a measure of the accuracy of estimation of a factor F, by means of 


its regression on the variables, the coefficient of multiple correlation R,may 
be computed. This is given by 


(12.28) RE = Ваһ, + Bootes +... + В. 


The multiple correlation can be calcul 
Table G.5, by multiplying the 8's by t 
given factor, summing, 
cipal factor 


ated directly from the values in 
д he first entries in each block for а 
and taking the square root. Thus, for the first prin- 


Re, = V.177(.858) + .314(.849) +... + .069(.619) = V.96277 = .981 . 


The multiple correlation coefficients for the other factors can be calculated 
in a similar manner and are given by 


Re, = 941, Е, —.980, апа R, = 961. 
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С.Б. Application to the Calculation of the Inverse of a Matrix 

In the shortened method of estimation of oblique factors the inverse of 
the matrix of correlations among the common factors is required. The solu- 
tion of this problem can be accomplished by the methods of this appendix. 
For simplicity, suppose there are three common factors and let the matrix 
of their intercorrelations be denoted by 


1 тіз тз | 
$ —|Tn 1 raj. 
Ты Тө 1 


From the definition of the inverse of a matrix, it follows that 


(G.13) 997 =I. 


If the elements of the symmetric matrix $7! are denoted by e's, equation 
(G.13) may be written in full as follows: 


l ns ru en біз біз 100 
(6.14) Та l resi} t| em ez е1 = |0 1 0}, 
ira ra 1 Өзі €32 Єзз 0 0 1 


in which r;, = rj; and ед = еу. The problem is to determine the elements 
of the inverse matrix from the known correlations. 

The matrices in the left-hand member of equation (G.14) may be mul- 
tiplied and the resulting nine elements set equal to the corresponding ele- 
ments of the identity matrix on the right. This yields nine algebraic equa- 
tions for the solution of the nine e's (the symmetry of the matrix serving as 


a check on the calculation of the elements off the diagonal). These equa- 
tions may be grouped in the three sets: 


1l + en + rise + ren = 
Taen + 1+ en + T2331 


1 

(G.15,) 0 
Таба + Tas +1+e3 = 0 
0 

1 

0 


Ш 


l- e2 Тоба + Tiges = 
(G.155) T€ + 1 + 629 + Тәзе, = 
Talig + T3222 +1 + eg; 


l- eis + тез + Тузезз = 
(С.15з) Talis + 1+ 623 + 7әҙбіз = 0 
731€13 + T32€23 Tli 


| 
e 


І 
- 
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It will be noted that each set of three equations involves the same matrix 
of coefficients—namely, the correlation matrix $. Hence, the work сап be 
во organized that the solution for all the e's can be made simultaneously. 

То illustrate the procedure for caleulating the inverse of a matrix, а 
very simple example will be employed. Consider the following matrix of 
intercorrelations of the two oblique factors, y; and уз, from the analysis of 
the eight physical variables: 


ф = || 1.000 71) 
= || 484 1.0001 


The inverse of this matrix сап be computed by means of Gauss's method 
of substitution. Construct Table G.6, putting the elements of $ (excluding 
those below the diagonal) in lines 1 and 3 of the left-hand portion, and the 
identity matrix in the right-hand portion of the main body of the table. 


TABLE G.6 
CALCULATION OF THE INVERSE OF A MATRIX 
Line Directions 2а | Үз Identity Matrix Check 
T. esses dens Түү 1 2484 1 0 2.484 
аана -L, —1 — .484 - 0 -2.484 
E teams Tk [ә L. 0 1. 2.000 
ЫЗА caus duci ewe. xj — .284 — .484 0 — ,718 
Discs estia +1. |... .766 — .484 d. 1.282 
—— —Ls/.766 | ТЕТЕ —1. .632 —1.305 —1.673 
Inverse 
From L; and L,: Calculate e and 6s а 1.306 - 
From 14: Caleulate e; and beer "үне - .682 n s петно: 


The calculations аге similar to those in the preceding sections of this ap- 
pendix, with the left-hand portion identical to that in Table G.4. The order 
of recording the elements obtained in the “back solution” is reversed from 
that of the preceding examples, however. The calculations from the last 
line of the "forward solution," although performed first, are put in the last 
line of the “back solution." This is continued, in reverse order, until the 
calculations from line 2 of the “forward solution” are recorded in the first 
line of the “back solution." In this manner, the final inverse matrix is dis- 
closed in the natural order at the bottom of the table. 
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TABLE H.2 
STANDARD ERRORS OF GENERAL-FACTOR RESIDUALS 


А 
ы” 10 | .15 | .20 | .25 | .30 | .35 | .40 | .45 | .50 | .55 | .60 | .65 | .70 | .75 
ЕШ аад -843|.336|.327| .317.305,.292| .277|.261|.244| .925 .205/.184) .1611.138 


.258|.249|.238| .226).213|.190| .184|.167|.150| .132/.112 
.2241.215|.206) .196/.185].172| .159|.145|.130| .114/.097 
Қ .200|.193).184) .175|.165|.154| .142.130/.116/ .102/.087 
:194.189/ .183).176].168] .160).151).141] .130,.118/.106) .093/.079 


:180|.175| .169|.163).156) .148/.139/.130| .120/.110/.098) .086|.074 
-168}.163) .158/.152/.146) .138/.130/.122| .113).103|.092, .081/.069 
.158/.154| .149|.144|.137| .131|.123|.115| .106/.097/.087| .076|.065 
-150|.146| .142|.136].130) .124|.117|.109| .101|.092!.082| .072).062 
-143|.139] .135).130}.124) .118|.111|.104| .096/.087/.078) .069|.059 


.187/.133| .129|.1241.119| .113).107|.099! .092/.084/.075/ .066/.056 
.132/.128| .1241.120|.114| .109|.102|.096| .088/.080|.072/ .063|.054 
-127.124| .120|.115).110| .105|.099|.092| .085|.078|.070| .0611.052 
-123).119) .116).111).106} .1011.095 .089) .082|.075].067) .059!.050 
-119).116) .112|.108|.103) .098/.092|.086| .080).073].065) .057|.049 


:115|.112| .109].105}.100} .095|.090|.084| .077.070|.063| .055|.047 
:112|.109) .106/.102|.097| .092/.087|.081) .075 .068|.061] .054|.046 
- 109}. 106) .108).090/.095) .090/.085 .079/ .073|.067|.060 .052!.045 
- 106}. 103) .100].096).092) .088).083).077| .071/.065|.058. .051|.044 
-095|.092/ .090].086).082) .078).074).069|. 064/.058.052| .046|.039 


-087|.084) .082|.079).075) .071|.067,.063| .058).053].047| .042|.036 
-080/.078| .076|.073).070/ .066/.062|.058| .054/.049/.044| 039.033 
:075|.073| .071).068|.065) .062/.058 .054| .050 046/.041| .036|.031 
:071|.069 .067..064..061| .058 .055'.051 .047|.043|.039, .034|.029 
-067|.065) .063/.061|.058| .055|.052/.049| .045/.041|.037, .032/.028 
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TABLE H.3 


SUPPLEMENTARY TABLE FOR COMPUTING 
STANDARD ERRORS OF RESIDUALS 


с Boi 2p} р; 2 — р; ої + 203 
.467804 .989536 
.456128 .969624 
.444000 . 949608 
. 431432 ‚929500 
.418436 .909312 
.405024 ‚889056 
‚391208 ‚868744 
‚377000 ‚848388 
‚862412 . 828000 
.847456 807592 
‚332144 ‚787176 
‚316488 ‚7667164 
- 300500 . 746368 
. 284192 . 726000 
. 267576 . 705672 
‚250664 ‚685396 
. 233468 ‚665184 
‚216000 ‚645048 
.198272 ‚625000 
. 180296 ‚605052 
‚162084 ‚585216 
‚143648 ‚565504 
. 125000 ‚545928 
‚106152 ‚526500 
‚087116 ‚5072382 
‚067904 .488136 
‚048528 .469224 
.029000 .450508 
.009332 .432000 
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TABLE H.4 


THE PROBABILITY INTEGRAL: AREA UNDER THE NORMAL 
CURVE IN TERMS OF DEVIATES FROM THE MEAN 


ЕТЕРІ 
Жазка сы» f^. an du 
470 


Ута 
| 
т z z z z 
f % z m : m : m : к 
| 
.00....| .0000 .80..| .2881 | 1.60..| .4452 | 2.40..! .4918 | 3.20..| .4993 
.02....| .0080 .82..| .2939 || 1.62..| .4474 || 2.42..| .4922 || 3.22..| .4994 
404... | .0160 .84..| .2995 | 1.64..| .4495 | 2.44..| .4927 || 3.24..| .4994 
.06....| .0239 .86..| .3051 1.66..| .4515 || 2.46..| .4931 || 3.26..| .4994 
408... .0319 .88..| .3106 || 1.68..| .4535 || 2.48..| .4934 || 3.28..| .4995 
.10....| .0398 .90..| .3159 | 1.70..| .4554 | 2.50..| .4938 || 3.30..| .4995 
.12....| .0478 .92.. 1.72..| .4578 || 2.52..| .4941 || 3.32..| .4995 
.14....| .0557 .94.. 1.74..| .4591 || 2.54..| .4945 || 3.34..| .4996 
.16....! .0636 ЖӨ. „| d 1.76..| .4608 || 2.56..| .4948 || 3.36..| .4996 
.18....| .0714 .98..|. 1.78..| .4625 || 2.58..! .4951 || 3.38..| .4996 
| 
.20....| .0798 || 1.00.. 1.80..| .4641 | 2.60..| .4953 | 3.40..| .4907 
422.22.| 10871 | 1.02... i| 1.82..| .4656 | 2.62..| .4956 || 3.42..| .4997 
:24....| .0948 || 1.04.. || 1.84..| .4671 || 2.64..| .4959 || 3.44..| .4907 
.26.... .1026 || 1.06. 1.86..| .4686 || 2.66..| .4961 || 3.46..| .4997 
.28....5 .1103 || 1.08. 1.88..| .4699 || 2.68..| .4963 | 3.48..| .4997 
230% aon} 1129 || 1,10... || 1.90..| .4713 || 2.70..| .4965 || 3.50..| .4998 
4925 хех 3955: || 1.19... i| 1.92..| .4726 || 2.72..| .4967 || 3.52..| .4998 
«Sisi ‚1881 | 1.14... 1.94..| .4738 || 2.74..| .4969 || 3.54..| .4998 
86....! .1406 || 1.16.. 1.96..| .4750 || 2.76..| .4971 || 3.56..| .4998 
:88....| .1480 || 1.18.. 1.98..| .4761 || 2.78..| .4973 || 3.58..| .4998 
:30....| .1554 | 1.20..| .3849 | 2.00..| .4772 || 2.80..| .4974 || 3.60..| .4998 
:42....| .1628 | 1.22..| .3888 || 2.02..| .4783 || 2.82..| .4976 || 3.62..| .4999 
.44....1 .1700 || 1.24..| .3925 | 2.04..| .4793 || 2.84..| .4977 || 3.64..| .4999 
46....! .1772 || 1.26..| .3962 | 2.06..| .4803 | 2.86... .4979 || 3.66..| .4999 
48....| -1844 || 1.28..| .3997 || 2.08..| .4812 | 2.88..| .4980 || 3.68..| .4999 
.50....| .1915 || 1.30..! .4032 | 2.10..| .4821 || 2.90..| .4981 || 3.70..! .4999 
.52....| .1985 | 1.32..| .4066 | 2.12..| .4830 | 2.92..| .4982 || 3.72..| .4999 
:94....| .2054 || 1.34..| .4099 | 2.14..] .4838 | 2.94..| .4984 || 3.74..| .4999 
:56....| .2123 | 1.36..| .4131 | 2.16..| .4846 || 2.96..| .4985 || 3.76..| .4999 
:58....| .2190 || 1.38..| .4162 | 2.18..| .4854 | 2.98..| .4986 || 3.78..| .4999 
.60....| .2257 || 1.40..| .4192 || 2.20..| .4861 3.00..| .4987 || 3.80..| .4999 
:62....| .2324 || 1.42..| .4222 || 2.22..| .4868 3.02..| .4987 || 3.82..| .4999 
:64....| .2389 | 1.44..| .4251 || 2.24..| .4875 3.04..| .4988 || 3.84..| .4999 
:66....| .2454 | 1.46..| .4279 || 2.26..| .4881 3.06..| .4989 || 3.86..| .4999 
:68....| .2517 | 1.48..| .4306 | 2.28..| .4887 || 3.08..| .4990 | 3.88..| .4999 
:20....| .2580 || 1.50..| .4332 | 2.30..| .4893 || 3.10..| .4990 || 3.90..| .5000 
:12....| .2642 || 1.52..| .4357 | 2.32..| .4898 | 3.12..| .4991 || 3.92..| .5000 
:24....| .2708 | 1.54..| .4382 | 2.34..| .4904 || 3.14. | .4992 || 3.94..| .5000 
:26....| .2764 || 1.56..| .4406 || 2.36..| .4909 || 3.16.., .4992 || 3.96..| .5000 
.78....| .2828 | 1.58..| .4429 | 2.38..| .4913 | 3.18.. .4993 || 3.98. | .5000 
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coordinate axes, 42 
coordinate hyperplanes, 42 
coordinates, 43 
definition of, 42 
general; see General Cartesian coordi- 
nates 
rectangular; see Rectangular Cartesian 
coordinates 
Center of gravity; see Centroid 
Centroid, 181, 240-41 
Centroid solution 
assumption on residuals, 181 
comparison with principal-factor solu- 
tion, 184 
examples, 189-92 
formula for coefficients of any factor, 
187 
formula for coefficients of first factor, 
182 
formula for coefficients of second factor, 
186 
geometric interpretation, 181 
method of calculation, 353-72 
as preliminary solution, 102 
reduced pattern, 249, 252 
theoretical development, 180-88 
Characteristic equation 
definition of, 161 


in determinant form, 160, 204, 206, 209, 
12 


in expanded form, 204, 207, 209, 212 
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Characteristic equation—continued 
roots of, 161, 163-64, 205, 210, 347, 373 
signs of coefficients in, 374 
Coefficient of multiple correlation, 270-71, 
273, 283, 285-86, 312, 391 
Coefficients of pattern identical with ele- 
ments of structure, 18 
Cofactor, 302, 311 
Combinations, notation for number of, 25 
Common factors, 4, 12, 59 
conditions for m, 77-78 
conditions for one, 68-71 
conditions for two, 72-76 
dependence upon set of variables, 107 
number of, 64-68 
theorem on number of, 59 
Common-factor space, 59-63 
determination of; see Chap. IV 
theorem on, 59 
Communality, 13-15, 20, 59, 313 
arbitrary estimates, 157-58 
assumption in calculation of, 115 
complete estimates, 158, 163, 188, 296 
determination of, 78-84 
estimation of, 150-59 
average of all correlations, 157, 192 
highest correlation, 157 
reducing to rank one, 157-58 
triad, 157 
as mean of triads, 115 
proportional to length of vector, 236 
residual, 162 
Complete index of factorization, 15 
Complexity of variables 
definition of, 16 
in preferred factor solutions, 106 
as a statistical standard, 90 
Components; see Factors 
Composite of matrices, 266 
Composite variables 
correlations, 35-38, 277 
correlations with factors, 241 
correlations with original variables, 38 
definitions of, 34 
in estimation of factors, 275-77 
in oblique solution, 241, 246, 249, 255, 
324-25 
standard deviation of, 35, 38, 241 
Composition of variables, 12-15, 288 


Contributions of factors, 13, 160, 163, 211, 
213, 226, 228, 247-48 
direct, 247-48 
joint, 247-48 
in relationships among factors, 292 
total; see Total contribution of factor 


Contributions of variable, direct, indirect, 
total, 271, 274 


Coordinate system; see Cartesian coordi- 
nate system 
Coordinates 
definition of, 43 
factor coefficients as, 59-60, 181, 216, 
235, 242, 246 
translation of, 46 
Correlation matrix, 19-20, 278 
approximate rank, 157 
cofactors of, 268 
of composite variables, 275 
о: абак, 250, 253, 257, 269, 281, 311, 
326 
inverse of, 275, 393 
notation for, 165 
power of, 167, 343-45 
rank of, 68, 76, 78-80, 110, 137, 150, 295 
of variables and factors, 207 
Correlations 
between composite variables; see Com- 
posite variables 
of factors, 237 
linearly independent; see Linear inde- 
pendence of correlations 
reproduced; see Reproduced correlations 
residual; see Residual correlations 
of sums; see Correlations of sums 
Correlations of sums, 34-37 
applications of, 37-38 
see also Composite variables 
Cosine of angle between lines 
and correlation coefficient, 58 
in general coordinates, 55 
in rectangular coordinates, 51-53 


Cramer’s rule; see Determinantal method 
for solution of equations 


Cureton, Edward E., 7-8 


Derived principal-factor solution 
from averoid solution, 212-14 
from bi-factor solution, 209-11 
from centroid solution, 206-8 
examples, 206-14 
labor of computation, 206, 212 
solution of polynomial equation in, 373- 
80 


theoretical development, 202-5 
transformation from initial solution, 
201-5 
Descartes's rule of signs, 373 
Descriptive variables, 3, 23 
Determinant 
n аи signs associated with elements, 


of coefficients of set of equations, 204 
cofactor, 302 

columns of, 301-2 

elements of, 301-2 
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expansion of, 77, 303 
minor, 302 
of observed correlations, 2607-68 
of order n, 302 
principal diagonal, 302, 382 
properties of, 74 
rank of, 305 
rows of, 301-2 
of second order, 301 
symmetric, 381-82 
of third order, 301 
Determinantal method for solution of equa- 
tions, 267, 271, 312, 381 
Diagonal matrix 
definition of, 307 
of unique-factor coefficients, 266, 279 
Юйден, L. E., 65, 160, 207, 212, 301, 317, 
374 
Direct factor solution 
assumption on residuals, 87 
calculation of factor coefficients, S6-87 
communalities, 86 
correlation matrix, 85 
illustration of, 85-88 
pattern plan, 85, 86 
Direct principal-factor solution; see Prin- 
cipal-factor solution 
Direction angles, 49 
Direction cosines 
conditions on, 218-19, 222 
definition of, 49 
of final reference axes, 218, 221 
of oblique reference axes, 239—42, 323-25 
of principal axes, 164, 202-5, 208, 210, 
384-86 
Direction numbers, 51 
Distance, 47 


Distinction between matrices and deter- 
minants, 304 


Doolittle, M. H., 381 


Doolittle method; see Gauss's method of 
Substitution 


Dot product; see Scalar product 
Doublet, 137, 339 
Dwyer, Paul, 286 


Efficient factor pattern, 238 
Eight emotional traits 


communalities, 176-77 
intercorrelations, 176 
principal-factor solution, 176-77, 342-52 


Eight physical variables 


approximative method estimation of 
oblique factors, 275-76 

B-coefficients, 80, 239 

bi-factor pattern, 169 


calculation of communalities, 80-83 
centroid solution, 192, 239 
complete estimation of oblique factors, 
271-74, 389-01 
complete estimation of principal fac- 
tors, 389-01 
composi*es of, 241, 275 
estimation of oblique factors by alterna- 
tive principle, 285-86 
intercorrelations, 80, 169 
means, 272 
multiple-factor solution, 223-26 
oblique factors estimated in terms of 
ideal variables, 287-88 
oblique solution, 242, 245 
derivation of pattern from structure, 
386-89 
inverse of matrix of correlations of 
factors, 393 
outline for calculation of oblique solu- 
tion, 237-48 
principal-factor solution, 168-73 
communality, 170 
naming of factors, 171 
pattern coefficients, 170 
rotation from bi-factor solution, 209- 
и 
rotation from centroid solution, 206-8 
shortened method estimation of oblique 
factors, 280-83 
standard deviations, 272 
values for two girls, 272 
Eight political variables 
communalities, 178 
description, 177 
correlations, 178 
oblique solution, 260-61 
principal-factor solution, 178-79, 260 


Equations 
characteristic; see Characteristic equa- 
tion 
polynomial; see Polynomial equation 
simultaneous linear; see Simultaneous 
linear equations 
Estimation of factors 
alternative principle method, 283-86 
Bartlett's principle, 283 
example, 285-86 
for common factors, 285 
principle of minimizing unique fac- 
tors, 283-84 
approximative method, 275-77 
complete estimation method, 267-75 
for all factors for all individuals, 268 
coefficient of multiple correlation, 
270-71, 285-86 
for common factors, 267-69 
employing pattern matrix, 269 
example, 271-74 
measure of accuracy, 270 
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Estimation of factors—continued 


in terms of observed values, 273 
for uncorrelated factors, 268 
for unique factors, 207—68 
shortened method, 278-83, 392 
assumption on residuals, 278 
assumption on uniquenesses, 278 
coefficient of multiple correlation, 283 
for correlated factors, 279-80 
example, 280-83 
for uncorrelated factors, 280 
in terms of ideal variables, 286-88 
Euclidean geometry, basic axioms, 41 
Examples; see Illustrative examples 


Factor 
bipolar; see Bipolar factor 
common; see Common factor 
doublet; see Doublet 
estimation of; see Estimation of factors 
general; see General factor 
relationships; see Relationships among 
factors 
specific; see Specific factor 
in standard form, 18 
unique; see Unique factor 
unreliable; see Unreliable factor 
Factor analysis 
applications of, 3-5 
basic data for, 11, 23, 295 
гш interpretation, 59-63, 159, 
14 
indeterminacy of, 23, 289 
nature of, 3 
object of, 289 
order of, 295-96 
and psychologists, 4 
reference system, 289 
and theory, 4, 7 
unity of, 289 
Factor coefficients as coordinates, 59-60, 
181, 216, 235 
Factor pattern 
adequacy of, 21-22 
coefficients interpreted as coordinates, 
59—60, 181, 216, 235, 242, 246 
definition of, 16 
determined from structure in oblique 
solution, 246-47, 386-88 
distinguished from structure, 234-37 
efficient orthogonal, 238 
notation, 16, 95, 266, 325 
as regression equations, 310-13 
see also Factor structure 
Factor solutions 
averoid; see Averoid solution 
bi-factor; see Bi-factor solution 
centroid; see Centroid solution 
combination of, 103 
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derived principal-factor; see Derived 
principal-factor solution 
direct; see Direct factor solution 
indeterminacy of, 23, 289, 313-15 
invariant, 107-8, 258 
multiple-factor; see Multiple-factor so- 
lution 
oblique; see Oblique solution 
principal-factor; see Principal-factor so- 
lution 
stability of, 153-54, 233, 248, 255, 258 
two-factor; see Two-factor solution 
types of, 8 
uni-factor; see Uni-factor solution 
Factor structure 
definition of, 16 
determined by transformation, 244—45 
distinguished from pattern, 234-37 
geometric interpretation of elements of, 
236, 243-44 
matrix form, 237, 266, 326 
see also Factor pattern 
Factors of mind, 4-5 
Filon, L. N. G., 123 
Fisher, R. A., 135 
Five physical variables 
B-coefficients, 140 
correlations, 139 
doublet, 142 
final bi-factor pattern, 142 
general-factor coefficients, 139 
general-factor residuals, 139 
modification of pattern plan, 140 
new pattern plan, 141 
standard error of general-factor coef- 
ficients, 140-41 
standard errors of residuals, 141 


Flat spaces, 42, 46 


Gauss's method of substitution, 271 
in calculation of inverse matrix, 392-03 


in derivation of pattern from structure 
386-88 


in derived principal-factor solution, 384— 
86 


in estimation of factors, 282, 284, 389-91 
outline for, 383 
theoretical development, 381-84 
General Cartesian coordinates 
elementary formulas in, 53-55 
in oblique solution, 236 
General factor, 4, 12, 107 
in bi-factor solution, 99, 106 
in principal-factor solution, 104, 106 
General rigid motion, 49 
Geometric fit 
ellipsoidal, 94, 104 
hyperplanar, 92-93, 102 
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linear, 91-92, 95, 99 
in multiple-factor solution, 223 
planar, 92, 99 
Geometrie interpretation of 
bi-factor plan ‚ т 
correlation, 5 
dimensionality 
elements of factor structure, 236-37 
factor analysis, 59-63, 159, 314 
indeterminacy of factor solutions, 314 
oblique solution, 234-37, 239 
pattern vs. structure, б 
statistical variables, 57, 92, 216 
two solutions of eight physical vari- 
ables, 171-73 
Geometry of higher dimensions, 41—43 
Girshick, M. A., 323 
Gosnell, Harold F., 177 
Group factor, 12 
in bi-factor solution, 101, 106 
in multiple-factor solution, 102, 106 
in uni-factor solution, 96-97, 106 


Groups of variables, 28, 96, 101, 295, 328- 
31 


Guttman, Louis, 278, 286 


Harman, Harry H., 18, 19, 34, 275, 278, 
289 


Hayford, J. F., 381 

Holzinger, Karl J., 18, 19, 34, 94, 101, 112, 
144, 274, 289, 309, 311-12 

Homogeneous coordinate system, 42 


Homogeneous linear equations of rank 
(1— 1), 161 

Horner's method, 377-79 

Hotelling, Harold, 5, 104, 155, 156, 159, 
163, 165, 210, 313 

Hyperplane, 41—43 

Hypothetical design, 23 


Ideal variables, 265, 286-87 
Identity matrix, 280, 293, 307, 392-93 
Illustrative examples: see under each of 
the following: 
Eight emotional traits 
Eight physical variables 
Eight political variables 
ive physical variables 
Thirteen psychological variables 
Twenty-four psychological variables 
Index of completeness of factorization, 15 
for twenty-four psychological variables, 
5 


Inner product; see Scalar product 
Intercorrelations of variables, 3 


Invariant lines (principal axes), 164 
Inverse matrix, 275, 278-81, 287, 290 
cofactors in, 308 
computation of, 287, 203, 308, 392-93 
definition of, 308 
of product of matrices, 308 


Jackson, Dunham, 133 
Kelley, Truman L. 
Kendall, M. G., 9 
Kronecker delta, 49, 219, 314 


Lagrange's multipliers, 203, 322 

Lamb, Horace, 108 

Laplace's development of a determinant, 
2512 


Law of cosines, 52, 54 
Ledermann, Walter, 65, 72, 278 
Linear combination and dependence, 43- 
16 
Linear dependence 
definition of, 43-44 
theorem on, 45, 316-17 
transitive law for, 46 
Linear equations; see Simultaneous linear 
equations 
Linear independence 
of columns of matrix, 318 
of correlations, 64-66 
definition of, 44 
of equations, 210 
of rows of matrix, 319 
Linear spaces, 42, 45 
theorem on, 46 


McMahon, James, 94 
Mathematical theory and observed data, 
6-7 
Matrix 
cofactors of, 268 
column vector, 266, 286, 290, 319 
correlation; see Correlation matrix 
definition of, 303 
definition of rank of, 305 
determinants of, 305-6, 317 
difference of, 306 
elements as rectangular Cartesian co- 
ordinates, 56 
equation, 279, 287, 291, 305 
identity, 280, 293, 307, 392-93 
inverse of, 275, 278-81, 287, 290, 293-94, 
308, 392-93 
multiplication of, 205, 343-45 
column-by-column, 280-81 
commutative law, 306 
row-by-column, 280, 294, 306 
row-by-row, 294 
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Matrix—continued 
nonsingular, 305, 308, 319 
notation, 19, 165-67, 217-18, 220-21, 
265, 303—4 
one-rowed, 304 
order of, 278, 287 
orthogonal, 49, 217-18, 222 
pattern; see Pattern matrix 
postmultiplication of, 227, 279, 294, 
306-7, 325 
premultiplication of, 278-79, 287, 290- 
91, 306-7 
principal diagonal, 280 
product by scalar, 307 
product by transpose, 58, 317-19 
rank of symmetric, 65 
of reproduced correlations, 19, 278 
diagonal elements, 20, 278 
residual, 166-67, 186-87 
scalar, 307 
singular, 305 
square, 303, 305 
sum of, 282, 306 
symmetric, 280, 287, 305, 328, 389, 392 
theorems involving rank of, 45, 46, 58, 
316-19 
of transformation, 205, 208, 210, 217, 
220-21, 244-45, 290, 294, 386 
transpose of, 19, 58, 280, 285, 291, 304, 
305, 308, 319 
triangular, 247 
Maximal linearly independent set, 318 
Меш value of certain ratios, 124-25, 320- 
1 
Miller, W. L., 198 
Minor, 302, 311, 318 
Moulton, F. R., 23 
Mullen, Frances, 80, 138, 168, 169 


Multiple-factor solution 

complexity of variables, 106, 223, 229 

examples, 223-33 

form of, 101-3 

geometric fit, 223-24, 229-30, 232 

rotation of preliminary solution, 102, 
180, 222-23 

stability of, 233 


Norm, 47 
Normal equations, 267, 270-71, 276, 389, 
391 
Normal law, equation of, 132, 397 
Normally distributed factors, 313, 315 
Notation 
B-coefficients; see B-coefficients, nota- 
tion 
coordinates of points, 43 
elements in different spaces, 59-63 
in Horner's method, 377 


matrix; see Matrix notation 

number of combinations, 25 

in principal-factor method, 348 
reflection of variables, 185 

for regression coefficients, 267 
set-theory; see Set-theory notations 
summational; see Summational notation 


N-tuple; see Point 


Oblique reference system, 235, 238 
Oblique reference vectors 
angle between, 236, 243 
correlation between, 237 
direction cosines of, 239-42 
variances of, 242 
Oblique solution 
angle between reference axes, 236 
comparison for different initial patterns, 
249, 258 
examples 
based upon bi-factor patterns, 255-59 
based upon centroid patterns, 249-55 
of inapplicability, 260-61 
geometric basis, 234-37 
outline for calculation, 237-48 
contributions of factors, 247-48 
direcHon cosines of axes, 230-42, 323— 


5 
factor pattern, 246-47 
factor structure, 243-46 
initial orthogonal pattern, 238 
intercorrelations of factors, 242-43, 
326 
subsets of variables, 238-39 
parts of, 237 
pattern derived from structure, 246-47, 
386-88 і 
rotation of orthogonal solution, 234, 237 
stability of, 248, 255, 258 
Order of rotations, 219, 222 
Orthogonal matrix, 49, 217-18, 222 
Orthogonal reference vectors 
in bi-factor solution, 117 
in centroid solution, 181 
in multiple-factor solution, 222 
in principal-factor solution, 202 
Orthogonal rotations; see Orthogonal 
transformations 


Orthogonal solution, 90-91, 181, 202 
Orthogonal system, 202, 220, 232 
total contribution of, 233 
Orthogonal transformations 
definition of, 49 
in higher-dimensional space, 221-22 
in matrix notation, 217 
notation in, 216-26 
order of, 219 
in plane, 215-18 
to principal-factor axes, 202-14 
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product of, 219 Же 
theoretical development, 215-22 
in three-space, 218-21 

Osgood, William F., 202, 322 


Parametric equations of line, 50, 51 
Pattern; see Factor pattern 
Pattern matrix, 19-20, 208, 326 
centroid, 206, 205, 225 : а 
common-factor portion, 266, 281, 256 
complete, 206 
definition of, 19 
in estimation of factors, 269 
principal, 205, 208, 211 
of uniquenesses, 281 
Pearson, Karl, 123 
Pencil of planes, 99, 117 
Pentad eriterion, 72 
Poinearé, H., 23 
Point 
arithmetic, 42 
geometric, 42 
Point representation of variables, 57, 93- 
94, 159 
Polynomial equation 
isolation of roots, 373-77 
sequence of, 375 
solution of, 373-80 
Postmultiplication, 227, 279, 294, 325-26 
Preferred forms of solution, 8, 265, 289, 
296 
assumptions and properties of, 106-8 
standards for judging, 89-94 
composition of variables, 90 
contributions of factors, 91 
geometric fit, 91-94 
parsimony, 90 
uncorrelated factors, 90-91 
Preferred System of reference, 89 
Premultiplication, 278-79, 287, 290-91 
Primary abilities, 102, 107 
Primary factors; see Multiple-factor solu- 
tion 
Principal components; see Pr 
solution 
Principal-factor solution 
assumption on residuals, 160 
complexity of variables, 106 
derivation of method, 322-23 
erived; see Derived principal-factor 
solution 
examples, 168-79 
form of, 103-5 
iterative method of calculation, 163- 
68, 342-52 
labor of computation, 174-75, 179, 206, 
212 


incipal-factor 


orthogonality of coefficients, 163 

theoretical development, 159-63 
Probability integral, 132, 397 
Product of matrices 

inverse of, 308 

rank of, 58 

transpose of, 308 


Product matrix, 221 

Product of rotations, 219, 221 
Projections of lines, 216-17 
Projective axiom, 315 
“Pythagorean” relation, 15 


Radius vector, 54 
Reciprocal of matrix; see Inverse matrix 
Rectangular Cartesian coordinates 
elementary formulas in, 47-53 
Reduced pattern, 242, 245, 255 
Reduced structure, 245, 276 
Reference axes; see Reference vectors 
Reference vectors 
centroid, 181 
direction cosines of, 164, 202, 208 
and linear fit, 92 
oblique; see Oblique reference vectors 
orthogonal; see Orthogonal reference 
vectors 
principal, 202 
unit, 235, 289 
Reflection of axis, 226, 227 
Reflection in the origin; see Reflection of 
variables 
Reflection of variables, 105 184-88, 295 
361-66 m ed 
Regression coefficients, 17, 267, 311 


Regression equations, 17, 265, 976- 
311-13 PER 

b between pattern and struc- 
ure 


geometric distinction, 234-37 
matrix formulation, 17, 266, 269, 325-27 


Relationships among factors, 289-92 
example, 292-95 


matrix formulation, 291 

transformation of coordinates, 292, 294 

used in estimating factors, 292 
Relative contributions of factors, 91 
Reliability, 14-15, 295 
Reproduced correlations, 19, 21 

from bi-factor solution, 101 

matrix of, 19, 278 


related to observed correlations, 21, 60, 
265 


аз scalar products, 61 
from two-factor solution, 112 
from uni-factor solution, 97 


416 FACTOR ANALYSIS 


Residual correlations 

definition of, 21 

final, 121, 136 

first-factor, 162 

general-factor, 120, 135 

general notation, 162, 187 

matrix of, 166-67, 186-87, 335 

standard error of, 128-32 
Residual-factor space, 120, 183, 187 
Residual vectors, 184 
Residuals; see Residual correlations 
Rotation; see Transformation 


Sampling errors 
approximations to, 129-31, 394-96 
of communality, 123-24 
of factor coefficient, 122-32, 147-48, 
226, 394 
interpretation of, 132-36 
in judging factor solutions, 111 
of residual, 128-32, 149, 395-96 
of triad, 122-23 
Scalar, 325 
definition of, 307 
product by matrix, 307 
Scalar matrix, 307 
Scalar product of two vectors, 47, 53, 181, 
183, 202 
Scatter diagram, 57 
Schmidt, Margaret, 177 
Scientific parsimony, 3 
Scientific theory, parts in, 6 
Sedecimant, definition of, 252 
Set-theory notations 
definition of, 96 
illustration of, 97 
use of, 113, 115, 118-21, 321, 328-30 
Simple structure, conditions for, 102 
Simultaneous linear equations 
solution by determinants, 267, 271, 312, 


381 
solution by Gauss's method of substitu- 
tion, 381-93 


Sisam, C. H., 219 

Snyder, Virgil, 219 

Sommerville, D. M. Y., 41 

Spearman, Charles, 4, 34, 68, 98, 101, 107, 
111-12, 167, 296, 309 

Specific factors, 14-15 

Specificity, 14-15 

Standard error of function in terms of 
standard errors of variables, 319-20 

Standard error of a variance, 82 

Standard errors; see Sampling errors 


Statistical variables 
design of, 23, 83, 261, 295 
interpreted as vectors, 40, 57 


as linear functions of factors, 12, 16, 59, 
60, 265 
observed values, 10, 265 
point representation, 57 
in standard form, 11, 13 
standardized values, 11, 273 
vector representation, 57 
as vectors projected into common-factor 
space, 60, 235 
Structure; see Factor structure 
Sturm's method, 374 
Sturm's theorem, 375-76 
Summational notation, 10, 25, 54, 96, 113, 
353 
Swineford, Frances, 101, 144, 274, 309, 328 
System of elements, 96, 117 
System of numbers; see Point 


Tests of significance, 132-36 
Tetrads, 4, 68, 112 


Thirteen psychological variables 
averoid solution, 194-97 
bi-factor solution, 153-54 
centroid solution, 189-90, 353-72 
correlations, 30 
multiple-factor solution, 226-20 
intermediate coordinates, 227 
naming of factors, 229 
ko pin of factor system, 
oblique solution based on bi-factor pat- 
tern 
contributions of factors, 258 
correlations of factors, 257 
direction cosines of axes, 256 
factor pattern, 257 
factor structure, 257 
oblique solution based on centroid pat- 
tern 
angles between axes, 250 
contributions of factors, 251 
correlations of factors, 250 
direction cosines of axes, 249-50 
factor pattern, 251 
factor structure, 251 
reduced bi-factor pattern, 256 
reduced centroid pattern, 249 
relationships between bi-factor and 
oblique solutions, 292-95 


Thomson, G. H., 283 


Thurstone, 1.. L., 5, 19, 22, 34, 59, 94, 102, 
107, 150, 158, 180, 184-85, 188, 215, 
309 
Total contribution of factor, 13, 160, 162, 
228, 248, 322, 352, 376, 380 
Transformation 
linear, 48 
matrix, 205, 208, 210 
in oblique solution, 243 
orthogonal, 49, 202-5, 215-22 
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іп principal-factor solution, 163-6. 
201-5 
of scale, 273-74 
theorem on, 49 
Transpose of matrix, 280, 285, 291, 305 
definition of, 19, 304 
notation for, 304 
theorems on, 58, 308 
"Triads 
in bi-factor solution, 120-21, 337-39 
definition of, 71 Е 
in estimation of communality, 115, 157 
standard error of, 122-23 
in two-factor solution, 112, 114-15 
Tucker, Ledyard, 19 
Twenty-four psychological variables 
appreximative method of estimation of 
i-factors, 276-77 
averoid solution, 198 
B-coefficients, 32 
bi-factor solution, 143-53, 32841 
communality, 150 
comparison of factor coefficients, 146 
doublet, 149 
naming of factors, 151-52 
new pattern plan for six variables, 145 
pattern, 144 
specificity, 150 
standard errors of coefficients, 147—48 
Standard errors of residuals, 149 
uniqueness, 150 
unreliability, 150 
centroid solution, 190-91 
composite variables, 37-39 
correlations, 30 
derived principal-factor solution, 212-14 
Ww m cosines of principal axes, 384— 
6 
first two principal factors obtained di- 
rectly, 174-75 
groups of variables, 33 
means, 29 
multiple-factor solution, 229-33 
intermediate coordinates, 231 
level distribution of variance, 231 
naming of factors, 232-33 
orthogonal systems, 232 
total contributions of factor Systems, 


oblique solution based on ы 
tern 
contributions of factors, 259 
correlations of factors, 257 
irection cosines of &xes, 256 
factor pattern, 259 
factor Structure, 259 


о lique solution based on centroid pat- 
ern 


contributions of factors, 254 
correlations of factors, 253 


-factor pat- 


direction cosines of axes, 253 
factor pattern 4 
factor structure, 254 
reduced uni-faetor pattern, 
reduced bi-factor pattern, 
reduced centroid pattern, 
reliability coefficients, 29 
roots of characteristic equation, 374-80 
standard deviations, 29 
Two-factor solution, 98, 101, 111-15 
method of triads, 114-15 
Spearman's method, 112-14 
Two-factor theory, 4, 112 


Uncorrelated factors, 13 
Uni-factor solution 

complexity of variables, 106 

form of, 95-98 

ideal pattern, 97 

reduced, 255 

schematic representation, 96 
Unique factor, 4, 12, 266, 283 
Uniqueness, 13-14, 313 

in shortened method of estimation of 

factors, 278 

Unit reference vectors, 235, 289 
Unitary Traits Committee, 94 
Unreliable factors, 14-15 


Variables 
composite; see Composite variables 
descriptive; see Descriptive variables 
statistical; see Statistical variables 
Variation of sign, 373-76 
s representation of variables, 57, 91— 
Vectors 
addition of, 43 
column, 266, 286, 290, 319 
cosine of angle of inclination, 58-61 
linear combination of, 43 
multiplication by numbers, 43 
projected into common-factor space, 60 


reference; see Reference vectors 


unit, 235, 289 
zero, 319 
Wenger, M. A., 309 
Whee to stop factoring, 151, 158, 188, 194, 
4 
see also Adequacy of factor solution 
Whitehead, J. H. C., 48 
Wilcox, L. R., 96 
Wright, T. W., 381 
Yule, G. Udny, 93, 315 
Zero vector, 319 
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